Yeah the V2 departures endpoint has been down since sometime yesterday. Looks like an upstream connection is gone? Not clear whether it's this project or the actual underlying API that has the problem. Anything we can do to help?

https://status.transport.rest/779961406

I can run this server locally and the departures endpoint seems fine, so that makes me think the problem is not upstream. Maybe just needs restarting?

curl http://localhost:3000/stations/900000120004/departures | jq .[0].journeyId

"1|31739|0|86|20062018"

The upstream endpoint has often been down for very brief periods of time in the past days.

I can run this server locally and the departures endpoint seems fine, so that makes me think the problem is not upstream.

I can confirm this. Seems like they have blocked the IP address of my server or something. Will look into it later.

Any news on this?

I requested a new IPv4 address for my VPS. Would like to avoid to moving the API over to another VPS.

If someone wants to host the endpoint or a fallback, let me know!

Still waiting for an IP address and currently don't have the time to migrate to a different VPS/VPS provider.

Again: If anyone is interested in hosting it, let me know! A really small VPS with nginx and Node.js installed is sufficient. @juliuste are you interested?

I can do it.

Endpoint should be up again.

Closing for now. We should find a long-term solution though for what I think is the cause: The VBB can block the IP address of the API server.

Thanks for solving this. I'm happy to contribute some € to the running costs. Let me know if that's possible.

@juliuste Do you have a PayPal/Patreon/OpenCollective/bank/crypto currency account?

You can support me on my Patreon page or via SEPA if you want to skip the fees. Keep in mind though:

Recently, I've started to work at Deutsche Bahn. I'm aware that I'm in a privileged position. Nevertheless, the sector of public transport is a place where open source work like mine is often not valued, let alone seen as necessary. Especially connecting public transport networks, for the benefit of everyone, is something that almost no company pursues. Independent support will allow me to keep working on what I – and assume you as well – think is truly important for passenger-friendly, accessible public transport services.

BTW @joecorcoran what are you building with the API?

Thanks for solving this. I'm happy to contribute some € to the running costs. Let me know if that's possible.

@juliuste Do you have a PayPal/Patreon/OpenCollective/bank/crypto currency account?

Thank you very much, but I'm alright, just support @derhuerst if you want to. 🙂

@derhuerst how much traffic would a VPS generate if I'd rent something to be a fallback?

@rejas Not a lot. For a while I've hosted it on a 3€ Scaleway VPS. It should be around 50k-100k requests per day, but vbb-rest is very light on the CPU, it basically just parses JSON.

Not sure how one would do a fallback for this though without adding too much tooling. Maybe a load balancer that supports an /health HTTP route.

I will also investigate if vbb-rest runs out of the box on Now. Then we'd have one fallback endpoint for free.

BTW @joecorcoran what are you building with the API?

I'm building a departure board app, designed for leaving open on a screen. Taking it slowly as I've never used React before. :)

Have deployed a fallback endpoint at https://vbb-rest-ribczgxrzz.now.sh. We still need to figure out how to do the load balancing though.

Seems like the new endpoint has been blocked as well. @derhuerst we could use a load balancer + different now instances, for example…

@joecorcoran @rejas @berlincount can you deploy vbb-rest via Now? With a (free) account, run now in the vbb-rest repo and post me the URL it returns.

• https://vbb-rest-usfolnrykj.now.sh
• https://vbb-rest-ribczgxrzz.now.sh

Didn't get it to work using nginx, trying haproxy now. If someone knows any other simple and efficient load balancing solution, let me know.

Seems like the vbb endpoint is working now.

Seems like the vbb endpoint is working now.

Can't reproduce. 😕

curl 'https://2.vbb.transport.rest/stations/900000013102/departures'

@derhuerst, try https://2.vbb.transport.rest/stations?query=oranienburg

Hm, it seems like half of the features of the API are working again.

I just discovered dfb6c4a. Does this mean, we will have a new working endpoint for VBB data? Are the rate-problems with VBB solved?

As mentioned above, I introduce breaking changes in hafas-client or any of its wrapper libraries (such as vbb-hafas or db-hafas) every now an then. When doing this, I increase the major version number by 1.

I've recently released hafas-client@4 and – building on top of that – vbb-hafas@6, db-hafas@4, bvg-hafas@2, etc. For the wrapping REST APIs, I've created a branch each for the old version (2 in vbb-rest, 2 for db-rest and 1 for bvg-rest) and adapted master to the aforementioned latest *-hafas versions.

Long story short: 3.vbb.transport.rest/3.db.transport.rest/2.bvg.transport.rest represent the latest data format based on hafas-client@4. I haven't announced these APIs publicly, so expect sudden changes.

Does this mean, we will have a new working endpoint for VBB data?

With the new 3.vbb.transport.rest API the fundamental problem is not solved: The VBB endpoint may still block the servers' IP address(es) if there are too many requests.

We can reduce the chance by load-balancing over multiple servers though, so please deploy the latest REST APIs on your servers or cloud services (such as DigitalOcean, Scaleway, AWS, Now or Heroku):

1. Check out vbb-rest#3/db-rest#3/bvg-rest#2.
2. Deploy it and check its health with the /health HTTP route (e.g. curl 'https://my-vbb-rest-3-instance.example.org/health).
3. Post its URL here.

Alright, thank you for clarifying.

I guess I was a little confused by the 3 in 3.vbb.transport.rest I thought that you maybe had a talk with the VBB people... But yeah, your explanation makes sense :)

3. Post its URL here.

Hi, I can offer these for the lb:

https://db.fishlevel.com
https://vbb.fishlevel.com
https://bvg.fishlevel.com

The instances run on a bare-metal server located in a data center in Berlin.
I could also add the version prefix in the domain if you prefer.
Do you know how much traffic is generated on average per month?

@jonathan-reisdorf

Thank you! 💚

I could also add the version prefix in the domain if you prefer.

Yes please, it helps me not to misconfigure something by accident. You could adopt the naming scheme of @juliuste and me (Julius will run b.*, I run a.*):

• c.3.vbb.fishlevel.com -> derhuerst/vbb-rest#3
• c.2.vbb.fishlevel.com -> derhuerst/vbb-rest#2
• c.2.bvg.fishlevel.com -> derhuerst/bvg-rest#2
• c.1.bvg.fishlevel.com -> derhuerst/bvg-rest#1
• c.2.db.fishlevel.com -> derhuerst/db-rest#2
• c.1.db.fishlevel.com -> derhuerst/db-rest#1

Do you know how much traffic is generated on average per month?

All 6 endpoints together get receive ~50-500k requests per day. On my (really cheap) VPS, they usually contribute to a load of about 0.15.

@derhuerst ok, I have set these up and also c.3.db.fishlevel.com. The load is no problem, just the traffic might be, bc the data center where my server is housed has a fair-use policy regarding traffic (-> if it's "a couple of hundred GB" per month I might have to pay an additional fee), so let's see how much traffic is generated after a month :)

We can also define your nodes as fallback, so that they won't receive any requests if a.* or b.* are up.

👍 sounds great!

@joecorcoran @deg0nz @ferby09 @k-nut @berlincount @timaschew Do you happen to have the knowledge and time to set up mirror/load balancing nodes as well? I would appreciate it.

Sorry, I don't have capacities for this right now...

I've set up 3.vbb.transport.rest a while ago (forgot to let everyone know about this), and so far it has been running reasonably well, so please use this endpoint or 2.bvg.transport.rest. Unfortunately both have a slightly updated output format, so make sure to check their docs.

I'm going to close this issue now, as the VBB admins haven't blocked 3.vbb.transport.rest so far. Thanks for you patience!

I have opened a thread in the transport.rest meta-repo to discuss hosting further: public-transport/transport.rest#2

I have set up v5.vbb.transport.rest, the successor of 3.vbb.transport.rest. As usual, unfortunately the response format has changed slightly (to the format of hafas-client@5, make sure to check its migration guide).

I will keep 3.vbb.transport.rest running for a while, and announce its shutdown via RSS before.