Describe the bug
If a probe target VM is recreated quickly enough and ends up with a new probe IP, HTTP probe still uses the old IP.

Cloudprober Version
All versions of cloudprober are affected.

Context
HTTP probes use endpoint.Key() to determine targets that are active and ones need to be recreated
https://github.com/cloudprober/cloudprober/blob/master/probes/http/http.go#L524

endpoint,Key() is defined as Name + labels, excluding IP. https://github.com/cloudprober/cloudprober/blob/master/targets/endpoint/endpoint.go#L41

So if only VM IP changes, we end up reusing the same target but keep probing the old IP.

One simple fix is to modify endpoint.Key() to use the endpoint IP too. I will do some digging to see if that causes other complications. If so, another quickfix is to modify http.go to append IP to endpoint.Key().