Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add semi-dense matching support (LoFTR) #173

Merged
merged 14 commits into from
Oct 4, 2022
Merged

Add semi-dense matching support (LoFTR) #173

merged 14 commits into from
Oct 4, 2022

Conversation

Phil26AT
Copy link
Collaborator

@Phil26AT Phil26AT commented Mar 24, 2022

  • LoFTR module (hloc/matchers/loftr.py)
  • semi-dense matching on image pairs (hloc/match_dense.py)
  • quantization to store features and matches in hloc format
  • supports quantization to pre-extracted features (e.g. superpoint)

@Phil26AT Phil26AT requested a review from sarlinpe March 24, 2022 12:18
@reynoldscem
Copy link

reynoldscem commented May 27, 2022

I've had some good results using this. I was wondering though, if there is a reason the matching is hard-coded to a batch size of 1? It occurs to me that more fully utilising the GPU & maybe allowing data-parallel across multiple GPUs might make working with this a lot faster / more practical.

I'm also slightly confused about the use of conf.top_k. It appears to be referenced, but not assigned?

@reynoldscem
Copy link

Is this still in the pipeline to be merged?

@Phil26AT
Copy link
Collaborator Author

Phil26AT commented Sep 26, 2022

Yes this is still in the pipeline to be merged, and should be ready now. Tested on Aachen v1.1 with setup loftr_aachen (no superpoint keypoints), which should run on machines with 32GB RAM:

Reconstruction:
        num_reg_images = 6697
	num_cameras = 4331
	num_points3D = 6106347
	num_observations = 29844267
	mean_track_length = 4.88742
	mean_observations_per_image = 4456.36
	mean_reprojection_error = 1.31786

Localization results:

Day: 88.7 / 96.4 / 99.0
Night: 75.9 / 90.6 / 99.0

Re batch size: I tested it and for me it only works with a batch size of 1 on Aachen (RTX 3080, 6 pairs / second). Nevertheless, I put the match extraction using LoFTR in a separate for-loop in case someone is willing to make this multi-GPU compatible.

@sarlinpe sarlinpe merged commit 972b57f into master Oct 4, 2022
@sarlinpe
Copy link
Member

sarlinpe commented Oct 4, 2022

cc @JiamingSuen

@sarlinpe sarlinpe deleted the loftr branch October 4, 2022 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants