Quick post after smashing my head off the desk for 5 hours trying to understand YTF I couldn’t get Mongo 3.2 running in a Docker container on my Mac. Hopefully this post spares others the elastoplasts 🙂
Pitfall #1 – Mongo cannot write data to the host OSX machine
There are three factors that cause this issue:
- A lot of mongo/docker docs tell you to mount a local volume
- By default docker-machine uses the virtualbox driver to create a local VM
- Mongo uses mmap to turbo-charge access to files on disk
These facts led me (by way of this issue) to the fact that there’s been an ancient known issue on VirtualBox that basically says docker ain’t going to be able to read/write files with mmap.
So it’s pointless trying to tell mongo, via docker, to mount a local data volume, as the above bug means mongo isn’t going to be able to access it. (careful reading of the Docker Hub mongo docs may allow you to divine this, but it’s not at all obvious)
Solution – Create a data volume in the container rather than the host.
Like always once you understand the problem, the solution is simple. We can tell docker to create a data volume in the container rather than the host, as follows:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| docker run –name my-local-mongo -v mongo-data:/data/db -p 27017:27017 -d mongo |
In my (still) early days with docker I’d come across these cli commands which contained absolutely no description of WTF all the switches are doing, so a quick breakdown:
- –name my-local-mongo – This tells docker to give the container the name “my-local-mongo” rather than generating a name like tuftybeaver
- -v mongo-data:/data/db – This is the key switch, here we tell docker to create a new data volume called mongo-data and mount it as /data/db which is the default location mongo wants to write data to
- -p 27017:27017 – Here we tell docker to bind the container port 27017 to port 27017 on the host vm, allowing us to access docker locally.
- -d – This switch tells docker to run the process detached so we’re not running interactively.
If all has gone well docker should be running your mongo container successfully. A quick docker ps will show your running container.
docker logs -f my-local-mongo – will allow you to follow the log files
Pitfall #2 – I can’t connect to mongo from localhost
This is the classic docker school boy error, that everyone makes. The first time you do this you’ll be patting yourself on the back as you type localhost:27017 into your connection string and then wonder why the hell docker isn’t found.
This is because everything is actually running on your local docker virtual machine. To find the IP of the VM enter docker-machine ls
You should see the URL listed something like this tcp://192.168.99.100:2376 You need to snarf this IP for your connection string. The easiest way to sanity check that’s it all working is bash it into your browser, eg for this example:
The mongo database should helpfully respond with:
It looks like you are trying to access MongoDB over HTTP on the native driver port.
Demonstrating that your mongo db is running and responding to idiots trying to speak to it over HTTP 🙂
Pitfall #3 – No collections are displayed in RoboMongo
This was a real head scratcher, I was pretty sure everything was working, and yet RoboMongo wasn’t showing a damn thing. This was a classic case of me Sherlock Holmes style spending 2 hours eliminating the impossible, to find whatever remains, however improbable, must be the truth
So in a turn up for the books, there was no bug in my code!! It turns out by default Mongo 3.2 uses the WiredTiger storage engine, and RoboMongo can’t parse data stored in this format.
So instead if you’re using Mongo 3.2 you should use MongoChef to connect to your db. MongoChef will show you all your lovely collections, living within your containerised Mongo.
Hope the above helps, thanks 🙂
Convivial Coding 
Re: Problem #2, I’ve found that changing the Secondary Network adapter mode in VirtualBox gives you docker containers with their own DHCP address from your network (using bridged adapter in my case). This has been useful for sending data from hosts to monitoring dashboards or other reporting tools directly to the docker containers. The part I’ve struggled with, is that I believe the boot2docker iso image is by default configured to use host-only network in VB, so you have to change this by default every time you spin up the VB VM.
Hey Iain. What was the symptom you were seeing when launching the container pointing the volume to the shared folder? I am trying to do that with VMware AppCatalyst to see if the behaviour is different but when I launch the container with….
> docker run –name mongo -v /mnt/hgfs/dbpath:/data/db -p 27017:27017 -d mongo
(where /mnt/hgfs/dbpath is a shared folder on my Mac visible in the Docker host running on AppCatalyst)
…. it just exits.
If I try to launch busybox with …
> docker run –name busybox -v /mnt/hgfs/dbpath:/data -i -t busybox
… it just runs fine and from within the container I can see in /data the Docker host folder /mnt/hgfs/dbpath.
I am trying to understand if I am seeing your same problem (i.e. AppCatalyst has a similar limitation) or if it’s something else.
Thanks.
Massimo.
P.S. love your writing style. 🙂
Hi Massimo, sounds like a similar problem to the one I had. Virtualbox only has permission to write to directories living under /Users. Even if you chmod -777 /mnt/hgfs/dbpath it still won’t make any difference.
Try:
> docker run -name busybox -v /Users/massimo/data:/data -it busybox
Where you created a data dir under your /Users dir. Hope it helps and glad you like the blog 🙂
Hi Iain.
That’s what I have already done but it’s a multiple hope.
I have a folder on my laptop that is:
/Users/mreferre/mongo
I have then started a Linux VM on my laptop and shared this folder with this VM (via the AppCatalyst shared folders mechanisms) and it appears inside the VM as :
/mnt/hgfs/dbpath
If I run busybox with
> docker run –name busybox -v /mnt/hgfs/dbpath:/data -it busybox
It works just fine. From within busybox I can browse inside /data and if I create a file it shows up in “/mnt/hgfs/dbpath” in my Docker container VM (and hence it shows up in “/Users/mreferre/mongo” on my laptop since “/mnt/hgfs/dbpath” is a mount of “/Users/mreferre/mongo”).
However, if I try to run
> docker run –name mongo -v /mnt/hgfs/dbpath:/data/db -p 27017:27017 -d mongo
it just exits. If I try to run it interactively ( -it ) it says:
> chown: changing ownership of `/data/db’: Operation not permitted
Google is full of similar issues in vm environments trying to share folders with the host. E.G. : https://github.com/boot2docker/boot2docker/issues/587
Thanks for the post – it’s helpful!
FWIW – I’ve pretty much given up on VirtualBox (on OSX, at least) and now use only VMware Fusion: too many networking issues which, let’s be honest, I can’t see Oracle fixing anytime soon.
A quick note, though: if you use `docker-machine ip ` it will just give you the IP address, which if you’re bash-ing away can be useful, doing something like:
curl -d foobar http://$(docker-machine ip myhost):${PORT}/foo/bar
Pingback: 2016-01-19 : Bucket full of links! - Magnus Udbjørg
Just wanted to throw in my 2c. Using the docker native beta for OSX you can mount the data volumes to the local machine with no issues!
FWIW on OSX 10.11.5 + Docker native beta (1.11.1-beta12) you can create the data volume on the Mac. Bizarrely it gets the UID and GID of the running user + any sticky bits. (compared to native Docker on linux where the data volume and files get UID and GID 999)
Reblogged this on Hiten Pratap's Wonderful Technological World.