Measurement data optimized file format selection

Hello,

I’m looking for an open source file format for recording data measurement coming from multi channel oscilloscope (or any other analog/digital devices).
The goal is to create a standardized procedure of a test result, which could be used by any service in a society; from R&D, production, to after-market service.

Actually, I arrived here by looking for HDF5 file format, which was proposed by ChatGPT, and seems to correspond to my needs.
But, as I can see there are many others formats, and maybe there could be better than HDF5?

My constraints are :

  • record of all measurement channels (let say, around 20 channels)
  • each channel can have a any physical type (current/temperature/speed…)
  • channels can be analog or digital (usually, digital signals are recorded as analog signals and reprocessed when read by an external application)
  • size of each channel is not excessive. Usually, it is 1Msample per channel (meaning a file containing 20 channels = 20Msamples per file).
  • somme external informations are requested, like type of signal, name of the channel measured, scale (gain/offset), date-hour, and so on (let say around twenty pieces of information recorded as string)
  • The writing/reading (which must includes an user friendly HMI or non-developer user) code must be simple to develop and maintain/enhance by a beginner coding team.

Before, we were using CSV file format, but the recording were not so big and traceability constraints were less “agressives”. Now I have to upgrade our traceability system by recording more datas and retrieve them as fast as possible, and using smaller size files.

Hope it helps you to help me to select the best file format for my needs!

Best regards,
Guillaume

@Guillaume_Liagre

We’ve reviewed your requirements and found that HDF5 seems to be a strong (best) fit for your needs. However, we’re open to exploring other formats if you have any alternatives in mind. Feel free to share them, and we’ll investigate further.

1 Like

@shakeel.faiz

Hello,

Thank you very much for your answer!!