Archive | general RSS feed for this section

Using Fluentd and MongoDB serverStatus for real-time metrics

As developers, we often look for tools to make our work and processes more efficient. Sometimes we have to search for what we’re looking for and sometimes we’re lucky enough that it finds us! When our friends over at Treasure Data wrote to me about Fluentd, an open-source logging daemon written in Ruby that they created and maintain, I immediately saw value for MongoDB users looking for a quick way to collect data streams and store information in MongoDB.

Intro to Fluentd

Fluentd is an open source data collector designed to simplify and scale log management. Open-sourced in October 2011, it has gained traction steadily over the last 2.5 years: today, Fluentd has a thriving community of ~50 contributors and 1,900+ stargazers on GitHub with companies like Slideshare and Nintendo deploying it across hundreds of machines in production.

Fluentd has broad use cases: Slideshare integrates it into their company-wide infrastructure monitoring system, and Change.org uses it to route their log streams into various backends.

Most relevant to MongoDB developers, many folks use Fluentd to aggregate logs into MongoDB. The MongoDB community was one of the first to take notice of Fluentd, and the MongoDB plugin is one of the most downloaded Fluentd plugins to date. Continue Reading →

MongoDB and the new PHP on Heroku

Update 5/7/14 5:15PM: Post has been rewritten to reflect Heroku’s PHP Buildpack changes

Update 5/5/14 10:45PM: Heroku’s default PHP buildpack now allows Composer to install the MongoDB extension. You do not need to use the develop branch. To enable, add “ext-mongo” to the composer.json file as described in the Solution section.

One of our close Platform-as-a-Service (PaaS) partners, Heroku, recently announced its official public beta for new PHP features. This announcement was met with much excitement from the developer community, and the MongoLab team looks forward to working with PHP developers on Heroku who want to power their apps with fully-managed MongoDB databases.

This post will cover how to install the PHP MongoDB extension on Heroku. Continue Reading →

{ "comments": 1 }

Analyze MongoLab Data with Hadoop in Mortar

The following is a guest post by Doug Daniels, CTO of Mortar Data Inc.

Today, we’re excited to announce integration between MongoLab and Mortar, the Hadoop platform for high-scale data science. If you have one of the 100,000+ databases at MongoLab, you can now seamlessly use Hadoop to:

  • Run advanced algorithms (like recommendation engines)
  • Build reports that run quickly in parallel against large collections
  • Join multiple collections (and outside data) together for analysis
  • Store results to Google Drive, back to MongoLab, or many other destinations

In this article we’ll show you how to connect your MongoLab database to Hadoop, and then use Hadoop to do something simple but very useful: gather schema information from an entire collection, including histograms of common values, data types, and more. Mortar handles all deployment, monitoring and cluster management, so no prior knowledge of Hadoop is required. Continue Reading →

Announcing New MongoDB Instances on Microsoft Azure

The following is a guest blog post by Brian Benz, Senior Technical Evangelist at Microsoft Open Technologies, Inc.

Since the previous release of production-ready MongoLab plans on Azure, we’ve seen demand increase significantly. The MongoLab and Microsoft teams have been working together to develop a solution for your growing requirements and are excited to announce the arrival of our newest high-memory MongoDB database plans, with virtual machine choices that now provide up to 56GB of RAM per node with availability in all eight Azure datacenters worldwide. Continue Reading →

{ "comments": 5 }

MongoLab now manages over 100,000 databases! (102,280 to be exact)

We’re proud to announce that MongoLab is now powering over 100,000 cloud MongoDB databases in 23 datacenters worldwide! Continue Reading →

Finding duplicate keys with MongoDB’s aggregation framework

Quite frequently our users want to create a unique index on a data set but encounter some form of the following error because of duplicate key value(s):

E11000 duplicate key error index: db.collection.$field_1_field2_1  dup key: { : 1.0 : 1.0 }

While MongoDB supports an option to drop duplicates, dropDups, during index builds, this option forces the creation of a unique index by way of deleting data. If you use the dropDups option, MongoDB will create an index on the first occurrence of a value for a given key and then  delete all subsequent values. While this behavior may be acceptable in some cases, it’s important to be cautious whenever you are deleting data. Continue Reading →

{ "comments": 2 }

Future of MongoDB: Fireside chat with MongoDB CTO Eliot Horowitz

Last night I attended a Meetup at MongoDB Inc.’s new Palo Alto office to hear MongoDB’s CTO, Eliot Horowitz, speak about the product roadmap. With a new production release right around the corner and MongoDB World in the not-so-distant future, the buzz and excitement around all things MongoDB is high. For those who were not able to attend, we’re going to recap all the major points Eliot made.

Continue Reading →

{ "comments": 3 }

Managing disk space in MongoDB

In our previous post on MongoDB storage structure and dbStats metrics, we covered how MongoDB stores data and the differences between the dataSize, storageSize and fileSize metrics. We can now apply this knowledge to evaluate strategies for re-using MongoDB disk space.

When documents or collections are deleted, empty record blocks within data files arise. MongoDB attempts to reuse this space when possible, but it will never return this space to the file system. This behavior explains why fileSize never decreases despite deletes on a database.

If your app frequently deletes or if your fileSize is significantly larger than the size of your data plus indexes, you can use one of the methods below reclaim free space. Continue Reading →

{ "comments": 4 }