Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extracting in real time metadata #193

Open
Ruisute opened this issue Sep 1, 2024 · 11 comments
Open

Extracting in real time metadata #193

Ruisute opened this issue Sep 1, 2024 · 11 comments

Comments

@Ruisute
Copy link

Ruisute commented Sep 1, 2024

Hi, just wondering if this could help me to extract sensor data while streaming ¿It is possible to use the metadata to feed a control system?

Like an FPV drone while in flight (intensive use) or RC car that just need that data for state info and not to have a functional approach (partial use)

@dnewman-gpsw
Copy link
Collaborator

We have thought about this, but don't yet have public solution. GPMF timing does extract information from the MP4 wrapper today, whereas livestreaming isn't using MP4. Other then a small amount of precision loss in timing, for where would you hope to extract metadata? What data stream?

@Ruisute
Copy link
Author

Ruisute commented Sep 9, 2024

I would like to extract data from the sensors integrated at the GoPro (integrated IMU: gyroscope, accelerometer...) just to avoid using an external component.

My idea is to record from a GoPro camera and stream video, sensor data to a control hardware (some espressif chip) for my FPV project.

@dnewman-gpsw
Copy link
Collaborator

dnewman-gpsw commented Sep 9, 2024

Where would the data be extract from, technically? This is the issue this need to be determined.

@Ruisute
Copy link
Author

Ruisute commented Sep 9, 2024

From the telemetry that came with the MP4 (GoPro video metadata)

Sorry if maybe I didn't understand at all the question.

@dnewman-gpsw
Copy link
Collaborator

The problem is when streaming data, there is no MP4, so where to want to extract the telemetry from? There are no standards for this, something new is needed.

@Ruisute
Copy link
Author

Ruisute commented Sep 10, 2024

Do you know if there is another way?

@dnewman-gpsw
Copy link
Collaborator

I want to know where a developer using livestreaming would like to see the metadata stored, on a technical level. Otherwise this is too much hand waving.

@Ruisute
Copy link
Author

Ruisute commented Sep 10, 2024

And what about an non technical approach? Just to show a bunch of people how fast the pilot is going at a race, just for streaming purpose

@dnewman-gpsw
Copy link
Collaborator

It has to be implemented using technology. That is the problem. We have the data, just need to know which pipe to place it. Remember it will be up to a third party to extract it, so where would a third party expect to extract it? Livestreaming has a sender (GoPro can do) and a receiver (everyone else.) So the issue is on the receivers not having a standard place to store metadata.

@dnewman-gpsw
Copy link
Collaborator

In your original question: It is possible to use the metadata to feed a control system?
Sure, what control system, and where is it expecting to read telemetry data?

@realchrisolin
Copy link

It's looking like this ticket came from someone without a full grasp of the technical complexities at play here. As someone with a somewhat knowledgeable grasp of the issues at play (not nearly as knowledgeable as you or Daryl), I would posit that if this request is going to be taken seriously, the best solution would be to stream the data over IP/MQTT. The GPMF standard is open source, it's up to developers of relevant software to add real-time streaming support. My understanding is this functionality already exists for STANAG 4609, which is another (standardized) KLV format akin to GPMF.

https://standards.globalspec.com/std/1222789/stanag-4609
https://www.impleotv.com/content/klvstreamer/help/page_klv_in_t_s.html
https://github.com/impleotv/gstreamer-test-klv
https://www.vitec.com/fileadmin/downloads/Collateral/VITEC_MilitarySolutions__Brochure__22-E.pdf

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants