I think adding just rsync-pull would work! Vagrant 1.5 is pretty much wrapped up for features but I'd be happy to look at a pull to get it in in a 1.5.1 or something.

Maybe then vagrant rsync --pull?

Perfect.

If you'd like to try this functionality out, I made a gem that does (manual) pulls from the guest to the host. Check out https://github.com/smerrill/vagrant-rsync-back if you'd like to help test.

The gem itself is mostly implemented as a monkeypatch to RsyncHelper, so if people have good experiences with it, I can put together a pull request.

+1 for this feature.

I' ve also tested a little bit the vagrant-rsync-back plugin from @smerrill. Works fine so far on Arch Linux with Vagrant 1.5.4.

been working with vagrant-rsync-back by @smerrill (thanks) for a few weeks now, haven't noticed any issues so far with 1.5.2. Would be nice to have the watcher integrate this feature, it would improve the workflow a lot (stop watcher, rsync-back, start watcher).

For now we're using smth like this in our CI run script:

vagrant ssh-config > vagrant-ssh-config
rsync --archive -z --exclude .vagrant/ --exclude Vagrantfile --exclude .git --exclude tmp --exclude .bundle -e 'ssh -F vagrant-ssh-config' default:/vagrant/ .
rm vagrant-ssh-config

Would it be possible to have pull included two ways using vagrant rsync-auto?

Consider Unison: http://www.cis.upenn.edu/~bcpierce/unison/

I don't recommend using rsync for bidirectional synch. It is not a trivial task. Here is background info (or pattern) for doing master-master synchronization, in the context of databases, but it applies to files as well: http://msdn.microsoft.com/en-us/library/ff650702.aspx

You have to keep track of deleted files and conflicts, and that is what Unison does with 'shadow tables', only it keeps databases of the files in ~/.unison.

I've gotten around the problem of one-way rsync for commands that need to be executed in the guest but write files that need to be available on the host with

$ (cd /vagrant/project; rake db:migrate)

This changes to the default vboxfs (which is dog slow), does the command, writes to guest disk, synchs to host disk, then rsync will synch back to the /rsync synched folder.

+1 I couldn't agree more with @laurentlemaire, have been watching this issue since March 14 because it's a killer feature for web developers, hope it makes it to the next release of vagrant :) ; BTW i know this might not be the most appropriate place, however, a sincere thanks to @mitchellh for vagrant

Blarg, two-way sync worked so well (and automatically, without the need for rsync-auto) in 1.4.x. The current one-way sync just doesn't make any sense from a user perspective. I care little about performance when syncing, I just want to "vagrant up" and forget it.

Is there a way to revert to the previous shared-folder way of doing things?

If not, +1 to get back to some two-way syncing sanity.

@matthew-dean shared folder sync'ing wasn't removed, we are still using it.

This is a feature request for the people who can't/won't use shared folders because virtualbox has the worlds slowest implementation. Some of our devs see 1+ minute load times with vb shared folders.

(Personally, I'm using the Parallels provider, and shared folders are blazing fast.)

Hmm, ok. Maybe there's just something wrong with how I have it set up then, but syncing used to work before I upgraded Vagrant from 1.4.3 to 1.6.3. How do you use the parallels provider?

@matthew-dean https://github.com/Parallels/vagrant-parallels

If you happen to have a copy of Parallels, I highly recommend it. It blows away virtualbox.

@khromov It has not been removed; you may still use NFS or VirtualBox/VMware shared folders.

I use the following solution as a workaround to get a two-way sync with acceptable performance. First, use the standard VirtualBox shared folders and sync to /vagrant. Then use Unison inside the VM, to sync between /vagrant and /var/www. I made a simple shell script that runs the Unison sync every 3 sec.

<-VirtualBox shared folder-> /vagrant <-Unison file sync-> /var/www

Maybe it's not optimal but I see no other solution right now.

@jenmo917 That is an interesting solution, and it actually may help solve some of my problems. My biggest complaint with all current two-way syncing solutions is that they do not trigger OS file system events at the final destination in the VM (which I use to trigger build watchers like Grunt/Gulp). Rsync does a real write to disk, so the events are triggered, but it has been a real challenge to get it working two-way (and by challenge, I mean impossible). If your Unison solution works the same way as rsync at the final destination folder, then I may be able to use it.

Once question, is there a reason that you only run Unison every 3 seconds? Could it be run more frequently?

I've found that for front end development (i.e. sass, js, etc) that Virtualbox shared folders are plenty fast. The only problem as @duro points out is that they don't fire inotify events (and neither does Samba). I use my own build scripts and make sure to use regular file system polling to detect changes (e.g. with Watchdog use watchdog.observers.polling.PollingObserver). For frontend stuff where you only have say 50 files this is super fast and well under a second. I don't know if it's possible to change the type of watcher in Grunt or Gulp but that's what would need to be done.

Now for backend development where I have tens of thousands of files, shared folders and file system polling really starts to fall apart and can take anywhere from 3 to 60 seconds to detect changes (not to mention high CPU usage). If I'm doing a lot of this I just reboot and switch to rsync-auto.

I don't use Grunt but perhaps something like this alternate watcher is what people need: https://github.com/unbalanced/grunt-simple-watch

@duro I use 3 sec because it's enough for my purpose (PHP development in SugerCRM). When I save something in my IDE I have to wait approximately 3 sec until I can see the change in the web-browser. I tried to do the sync more frequently but I experienced issues with high CPU-load when I did that.

I've been using unison instead of the built-in vagrant support and it's been working well. This also lets me easily sync with Vagrants launched by other people and even production machines all with pretty much the same sync commands

@wysenynja and @jenmo917, have you guys tried using the vagrant-unison plugin?

https://github.com/mrdavidlaing/vagrant-unison

I have not, I'm just wondering if you have tested it.

@duro, yes I have, and I had problems with it. I think it was because it didn't have auto-sync and you had to trigger the sync yourself each time. I took the "on change mechanism" from vagrant rsync-auto and implemented it into the vagrant unison plugin. The problem I faced was that Unison was triggered on changes on the host machine, but not on the guest machine. When you think about it is quite natural because Rsync, where I found the code, is a one-way sync. This was how far i went with the unison plugin.

vagrant-unison can also only sync one directory. Additionally, I wanted to be able to easily sync to vagrants launched by other people and a vagrant plugin doesn't really make that easy

@dmatora:

1. does running vagrant plugin install vagrant-unison install your forked version? If not, how do we get yours installed?
2. Is there a way to change the ssh username to something besides 'vagrant' ?
3. I would have posted these questions on your fork, but issues are turned off

@flynfish no, it does't.
Simple way to install it would be to download https://www.dropbox.com/s/1yu1s7pr3qlr8ag/vagrant-unison-0.0.15.gem and do vagrant plugin install vagrant-unison-0.0.15.gem
but it's currently outdated.

I'm sorry to say that guys, but i moved to using nfs_guest, i tried to use it before unison, and nfs silently failed for me. But later i discovered than on OSX you need to have localhost in your /etc/hosts file in order for nfs to work and that solved it. With nfs_guest i don't have fs_events, but performance is close to filesystem, and i don't have sync failures, so i don't fix those anymore. Yet CPU load is so much smaller.

I have been using the vagrant-unison2 plugin with success. I was also using NFS but ran into issues with windows and npm.

I've seen a couple good suggestions here so far. I'm completely out of options because network-based shares don't work over VPN but our VPN client is not up for discussion which leaves openconnect out of discussion.

Rsync appears to be the only option as appallingly slow disk read and write speeds have ground our build and test processes down to a halt on the standard sync share.

But the issue with Rsync is that it only supports one way sync and of course requires cygwin to be installed as well. Bundling a two-way rsync into Vagrant would be a huge huge huge win.

It looks like there is hope though. I'm excited that this has been tagged as milestone 1.9.0 but 1.8.7 is only 33% complete as of this moment in time. An interim solution will absolutely have to be implemented.

Check out https://github.com/dustymabe/vagrant-sshfs/ /cc @dustymabe

@seantcanavan check out vagrant-sshfs - i've actually been thinking about proposing it for inclusion in vagrant itself. for now it lives pretty well on its own as a plugin.

Thanks for the ideas @purpleidea and @dustymabe. Unfortunately the ruby gem bundler uses a custom user agent string which the corporate firewall immediately shoots down which makes plugin installation impossible so I was hoping for a baked-in solution.

Looks like I gotta get the network guys on the phone...

Cheers.

@seantcanavan can you download the gem and install them directly (i.e. install from file instead of from rubygems.org) I think for vagrant-sshfs you need win32-process and vagrant-sshfs. if you get those two gems and then install directly from file (i.e. vagrant plugin install ./win32-process-X-X.gem && vagrant plugin install ./vagrant-sshfs-X-X.gem) then it should work

@dustymabe Finally got the gem files built for win32-process-0.8.3 and vagrant-sshfs-1.2.0. I created a gems directory at my project's root folder where the vagrant file is located and put both gem files in there. I executed vagrant plugin install ./gems/win32-process-0.8.3.gem from the root directory and received the error /usr/lib/ruby/2.3.0/rubygems/specification.rb:946:in all=': undefined method group_by' for nil:NilClass (NoMethodError)

EDIT01: Appears to be an issue with Ubuntu 16.04. Vagrant 1.8.1 and ruby 2.3.1.
EDIT02: Patched /usr/lib/ruby/vendor_ruby/vagrant/bundler.rb successfully. Directions here
EDIT03: New error after trying to install: Could not find gem 'win32-process (= 0.8.3)' in any of the gem sources listed in your Gemfile or available on this machine.
EDIT04: Appears Ubunbut 16.04 is kicking me in the teeth again... Investigating: #7493
EDIT05: Apparently need to upgrade Vagrant version. Command is "wget https://releases.hashicorp.com/vagrant/1.8.6/vagrant_1.8.6_x86_64.deb && sudo dpkg -i vagrant_1.8.6_x86_64.deb". Restarted my box and plugin install SUCCEEDED!
EDIT06: vagrant destroy now produces There was an error while trying to load the gem 'win32-process'. Gem Load Error is: unable to resolve type 'uintptr_t' I feel like I'm on a roller coaster.
EDIT07: I've filed an issue with the ffi plugin, which is a dependency of the win32-process plugin, here: ffi/ffi#530. I might have to open a different issue if the error message is a red herring and not really an ffi issue.
EDIT08: Installing win32-process on OSX and Linux totally breaks vagrant. Employing brain power to realize win32-process is a windows-only plugin solves the issue.
EDIT09: Getting a new error now on vagrant up: /opt/vagrant/embedded/gems/gems/vagrant-1.8.6/lib/vagrant/action/builtin/mixin_synced_folders.rb:137:inblock in synced_folders': Internal error. Report this as a bug. Invalid: sshfs (RuntimeError)` Double checking my syntax. Seems easy enough - like sshfs is an unknown synced_folder type.
EDIT10: Appears my locally built gem file is causing the error. When I manage to bypass the firewall and install remotely via the gem bundler it works fine. But more importantly success! Finally!

@seantcanavan wow - so you finally got something working?

@dustymabe Yes!! I hope I'm not derailing and someone else will find this info useful. My last question is this: I've pulled in vagrant-sshfs from outside our firewall once and installed it locally and it works great. Now I need to package it in a gem file to add it to our source so people inside the firewall can reliably retrieve it. I found the plugin installed at /home/username/.vagrant.d/gems/gems/vagrant-sshfs-1.2.0 but when I run bundle install inside the install dir I get

[!] There was an error parsing Gemfile: You cannot specify the same gem twice coming from different sources.
You specified that vagrant-sshfs (>= 0) should come from source at . and source at .
. Bundler cannot continue.

from /home/username/.vagrant.d/gems/gems/vagrant-sshfs-1.2.0/Gemfile:13

-------------------------------------------

group :plugins do"

gem "vagrant-sshfs" , path: "."

# Add vagrant-libvirt plugin here, otherwise you won't be able to

-------------------------------------------

Do you have a quick and easy way I can package your plugin to install locally via command line for future use?

@seantcanavan commenting out this line should work to fix that error: https://github.com/dustymabe/vagrant-sshfs/blob/master/Gemfile#L13

the other thing is - why not just download the gem from my releases page and use that?
https://github.com/dustymabe/vagrant-sshfs/releases/download/v1.2.0/vagrant-sshfs-1.2.0.gem

Hi there,

We would really like this, but this issue has been open for over a year with no one working on it, Leaving it open is unfortunately making our issue count look higher than it is. I’m going to close this and if someone wants to work on it I’d still be open to a PR!

Cheers!

Crap. I spent all this time and effort setting up Vagrant boxes in the hope of developing a suite of buildbots for cross-compiling systems language applications for different operating system kernels. And then I find out that rsync-based Vagrant synced folders--and there is no other option besides rsync-based Vagrant synced folders for the vast majority of guest operating systems--does not support bidirectional syncing. So there's little point in vagrant ssh -c 'clang -o /vagrant/hello hello.c' if the hello binary won't even be transferred to the host. :(

I have a few suggestions for implementing bidirectional rsync more quickly:

• https://github.com/deajan/osync , which claims to just depend on bash and rsync, and therefore could be easily installed automatically by Vagrant into rsync-enabled guests.
• Continue using /vagrant as a one-way source host->guest folder, and introduce a second folder like /vagrant-output that one-way syncs to the host as <host-cwd>/vagrant-output. When Vagrant rsyncs the /vagrant source folder, it should exclude vagrant-output.
• The user can also manually rsync files out of the guest to the host, but this represents a leaky API for Vagrant.

rsync configuration is something that Vagrant should really master, for more capable VM's. In the meantime, I might configure my buildbots to publish their artifacts to S3 instead of trying to copy the artifacts to the host :/

@mcandre would vagrant-sshfs work for you?

@mcandre I use vagrant-scp to retrieve artifacts from Vagrant machines.

@dustymabe vagrant-sshfs is interesting. Unfortunately, it is not clear that this plugin supports bidirectional sync in a single Vagrant box configuration--the documentation seems to indicate that either guest->host syncing can be configured, xor host->guest, but not both at once.

Also, this plugin does not support Windows, which for my purposes (cross-compilation) is a no-go.

Update

Good news, the vagrant-rsync-back plugin appears to work reasonably reliably, at least to-and-from UNIX-style operating systems like Debian, Illumos, and MINIX. Will be testing this with Windows as well. It stinks that vagrant rsync-back must be manually called in order to perform the other half of bidirectional sync, but at least it works! Would love to see this capability backported into the main Vagrant system so that rsync synced folders can work bidirectionally out of the box, so to speak.

@dustymabe vagrant-sshfs is interesting. Unfortunately, it is not clear that this plugin supports bidirectional sync in a single Vagrant box configuration--the documentation seems to indicate that either guest->host syncing can be configured, xor host->guest, but not both at once.

SSHFS is essentially a network filesystem. You can access files from the host or from inside the guest at the same time and they transparently update all locations. A longer description here.

Also, this plugin does not support Windows, which for my purposes (cross-compilation) is a no-go.

It does support windows, just not for reverse mounts (sharing guest files to a host folder), which I don't think you actually need. How about giving it a try so you'll know for sure 😃.

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.