San Francisco Ruby on Rails Meetup Tomorrow Night

Tomorrow evening is the Inaugural San Francisco Ruby on Rails Meetup so come out and meet your fellow SF Rubyists. Jonathan Siegel, founder of RightScale, RightSignature, and RightCart will be speaking. Be sure to sign up if you'd like to come. Pizza and Beer to be had by all!

Cloud + M2M = Disruption of Things

Part 1 of a white paper on Cloud and M2M is up on ReadWriteWeb. In it we mention one of our clients, Plaster Networks, makers of awesome IP-over-powerline network adapters. They use the cloud to make their products even more intelligent.

Here's the start:

The use of cloud infrastructure and cloud services provides a low-cost means to create highly scalable applications. Even better, the cloud dramatically improves development speed and agility. Applications can be developed in much less time with much smaller teams. And as these benefits extend themselves in the machine-to-machine (M2M) space, companies creating M2M applications will see dramatic reduction in the cost of developing applications and provisioning services.

Articles on the Internet of Things (or Web of Things) are increasingly finding their way into mainstream news. Executives of large companies (such as the CEO of Sprint) and even government officials (such as the Chinese Premier) are speaking about the possibilities and opportunities of having ubiquitous sensors connected to the Internet.

The use of the cloud - in combination with the advent of low-cost sensors and high-availability M2M data transmission - will transform old industries and modify many business models. Almost every major electronic device, vehicle, building component, and piece of equipment has the ability to become "smart" by connecting sensors to it. Most devices already do. The difference though is that moving data to the cloud and being able to process it in infinite combinations provides new capabilities in very low cost, transparent ways.

Read More -->

The Importance of Workers - Part 1

At Appoxy, every single one of our developers uses our new service SimpleWorker.com.  Some of them weren't sure of the value at first, but from the first time they queue()'d a nightly report builder worker, or schedule()'d the delivery of SMS's through Twilio, they were hooked on it's power.  It has become as integral a development tool as the database.

The best part?


Our developers don't need to know anything about queues (even though they do), or elastic servers, or runners, or schedulers, or even where our SimpleWorker servers live..... they simply know that the Ruby Compute Cloud is alive and waiting for their every request.


So why SimpleWorker?


First, we need to understand a bit about the current landscape of worker systems.  Here is a great writeup of Github's path through the various worker systems available today.  They ended up rolling their own.  Pay attention to how much effort they put into rolling their own, and how much potential effort any would-be user of Resque (or DJ, or any of them) has to go through to implement the system and infrastructure requirements.


https://github.com/blog/542-introducing-resque


Don't get me wrong, the current open source worker systems out there are fantastic, and each has a place in this world.  They have enabled hundreds if not thousands of websites to do amazing things.  But 95% of the world does not need to go through the trouble of rolling their own system and managing their own infrastructure.


In the next post, we'll talk about workers-as-a-service....

Chad

Cloud Computing Meetup tonight

Heading to a Cloud Computing Meetup in Mountain View tonight. Topic is go-to-market strategy and innovation. 300+ people have signed up.

Only issue with these types of talks is the definition of cloud computing is so broad (not unlike "hey, we're getting to together to talk about the Internet or Web 2.0"). With cloud computing now, topics run the gamut from SaaS to PaaS, enterprise b2b, consumer web, developer markets, tools, and beyond.

Now that we're into 2011, "cloud computing" needs subcategories or else it just looks like those darn Microsoft commercials about "going to the cloud."

Facebook to Email, a new Appoxy Production

We recently launched a new service at www.fb2e.com that will, with one click sign up, start sending your Facebook feed to your inbox every day. This is great for getting your social fix without having to go to Facebook all the time to see what's going on with your friends. Also great for those who work at a place that blocks Facebook. ;)

Check it out and let us know what you think in the comments below!

For the techies out there, the core part of this service is sending emails to a large number of people on a schedule and this was made easy by using SimpleWorker to schedule and send out all the emails. The Facebook connection part was made using the mini_fb gem.

"We're Going to Need a Bigger Boat"

The quote is, of course, from Jaws and is at the top of the list of top movie lines.

It's also a phrase that can easily be used in web app companies when it comes to data storage. Companies such as Google, Facebook, Twitter, and Zynga hit this issue early in their adoption curve because of the rapid growth of their users. It's why much of the development work on large scale data storage and analysis has come out of these companies (MapReduce, Hadoop, Cassandra, Hive, and Membase are just a few of the projects).

Almost any successful web application will run into this issue. At a prior company, we were collecting clickstream data for use in widgets on a webpage. A simple back of the envelope calculation says that collecting clickstream data across a network of media and retail sites quickly turns into a boatload of data. Its easy to say that only the most recent data has any real value -- which is true -- but historic data also has value, especially if you're trying to improve algorithms by running regression tests. Also, new algorithms applied to older data can provide valuable insights.

The rule of thumb at large Consumer Web 2.0 companies is that storage is cheap and data is valuable. Which translates into "store whatever you capture." A blog entry we did a few months ago on clickstreams, footstreams, and other datastreams touched on the value of this data.

As this notion of storing actions and status of people and things moves from Consumer Web to the Enterprise and the Internet of Things -- to encompass everything from the performance and status of cable modems, heart monitors, automobiles, jet engines, mining and agricultural equipment, shipping and manufacturing -- storing running streams of data will be an issue across sectors.

Which is where sharding comes in. Sharding refers to splitting data into managable chunks. It's used to refer to separating data across different databases. It's not a popular term outside certain tech cycles -- it still gets a lot of eyebrows and "what did you say". But look and listen for it more in 2011.

Travis and I had a great conversation before the holidays with Cory Isaacson, CEO/CTO of codeFutures which has a product called dbShards. It sounds like a neat product and definitely in line with where things are going. (It works with MySQL and many NoSQL databases.)

Image from codeFutures/dbShards

The conversation inspired Travis to add it to SimpleRecord, an Appoxy Ruby gem for connecting to Amazon's SimpleDB. We needed it for a client project -- handling sensor inputs from thousands of devices quickly fills up storage -- but also recognized it would be needed by others working with big data.

A week later we were speaking to the founder of a company making heavy use of SimpleRecord and SimpleDB. He mentioned he'd run into problems with data loads using SimpleDB. Fortunately, he'd seen the update Travis did to SimpleRecord. What he thought would be a few days of work to address turned in less than an hour using the new sharding capability.

Github page for SimpleRecord

There are many answers to getting bigger boats to store data. Sharding is but one of them. But it sure is nice to have simple solutions to what use to be complicated problems.

(And so yes, shark looks a lot like shard which plays into the Jaws reference and this classic scene.)

Automated Gawker DB Lookup/Reset Script

Due to the Gawker user database compromise, we thought we'd share the script we're using to reset passwords for those users that are found in the Gawker list. It's a Ruby script that will go through the list of compromised emails and check it against your database of emails.

You can find it on GitHub here.

Note: You will need the list of usernames and emails that were compromised in the Gawker hack. Due to privacy/security concerns, we are not including the username/email list (although it is widely available on the Internet). We know this is kind of a pain, but we don't want to be the ones spreading spam love around.

Included on the GitHub page for the script is a form to fill out if you need help or would want access to the Gawker list (provided we can verify you're who you say you are).