Changing databases is not a move to be taken lightly, especially when the switch is to a relatively new kind of database.
The Weather Channel, however, found that it had to switch to a NoSQL MongoDB data store in order to more quickly develop apps and add features to its range of Internet-based weather information services.
“There was an awful lot of work to do in wrestling the tools into doing what we wanted,” said Luke Kolin, vice president at The Weather Channel in charge of architecture. “There was just too much boiler plate and drudge work.”
For over a decade, the weather-oriented cable television channel had relied on a traditional enterprise IT stack, including Java Enterprise Edition, Apache Tomcat Java servlet containers and MySQL databases.
With this architecture, “we had taken it about as far as it could go,” Kolin said.
As part of an effort to update its IT infrastructure, The Weather Channel is now deploying the MongoDB NoSQL database to replace this stack.
The performance of the enterprise Java and MySQL stack was fine, but developing with them had grown increasingly difficult in the fast-paced competitive environment.
MongoDB provided the best combination of ease-of-development and fast response times.
As many other companies are learning, The Weather Channel is finding that its customer-facing technologies are among the most important to manage, given that such technologies are the primary interface for many of its customers.
IT analyst firm Forrester has estimated that by 2017, 31 percent of enterprise technology spending will be for products and services that interact with the organization’s customers, such as software for customer relationship management, Web content management, e-commerce storefronts, marketing automation and customer analysis. Spending for these technologies will grow by 10 percent each year, Forrester predicted.
As part of an overall rearchitecting of its media systems, The Weather Channel set up an instance of MongoDB Enterprise on the Amazon Web Services.
Earlier this year, it switched its iPhone and Android apps over to the new database and plans to move the rest of its Web operations to the service by the end of the month.
Currently, The Weather Channel’s MongoDB instance responds to 2 million requests every minute, on average, while maintaining user information and copies of the weather updates for tens of thousands of locations worldwide.
Replacing a relational database with MongoDB, or any other NoSQL database, has been a controversial topic over the past few years.
MongoDB is a NoSQL database, meaning that it sacrifices some of the advance parsing and joining abilities that SQL-based relational databases offer, in favor of offering the ability to scale across multiple servers so they can host very large data sets and respond to queries very quickly.
MongoDB is known as a key-value document store, meaning it emphasizes quick data storage and retrieval.
It some cases, this approach is not a good fit.
MongoDB was a bad choice for powering the ill-fated open-source Diaspora social network, developer Sarah Mei has claimed.
That project needed a relational data store, in which some data needed to be permanently linked to other data. MongoDB was best suited for storing documents that don’t need to be connected to one another in complex ways, Mei said.
In The Weather Channel’s case, however, MongoDB proved to be a good fit. “Our data access patterns were actually matching the NoSQL model pretty well,” Kolin said.
Weather forecasts for specific locations make up the vast majority of data the channel sends out. “Those access patterns suit themselves very well to key-value stores,” Kolin said. The company doesn’t have the vast Web of interconnected data that Diaspora had.
More importantly, if the structure of the data changes, then MongoDB’s schema can be easily adjusted, something that can not be done so easily in a relational database and associated software.
“Every time there was a simple, trivial change, we were having to modify five or six different components at three or four different levels of the stack,” Kolin said
MongoDB also brings The Weather Channel a much needed promptness.
The company has a set of alert services that send out warnings when severe weather is imminent. When a severe-weather warning is issued for a heavily populated area, such as Chicago, The Weather Channel must send alerts to the thousands of users who live in the affected ZIP codes, preferably within seconds.
The company tested a range of NoSQL databases, including Cassandra, Riak, CouchDB and MongoDB, as well as MySQL, to determine which one could most quickly filter a few thousand users from a much larger pool of candidates. MongoDB proved to be the best fit, able to deliver the information within seconds.
In addition to The Weather Channel, other companies that use MongoDB include MetLife, Bosch, Expedia, ADP and the city of Chicago.