Convert AudioData stream to arrays of floats, and convert arrays of floats to AudioData by lucasw · Pull Request #84 · ros-drivers/audio_common

x-raw allows my mic to work- needs to be parameter

lucasw

committed

46e1c7f

Not compiling yet, want to be able to decode audio and then process arrays of samples.

lucasw

committed

a740590

py example doesn't work

lucasw

committed

dba1563

This compiles but results in glib/gstreamer errors when it runs.

lucasw

committed

e830583

It looks like only one buffer is received, but I'm not handling it correctly so the callback gets triggered repeatedly with the same buffer. It seems like I need something else in the pipeline to …

lucasw

committed

2f2f32b

This may be the raw data- but what about 8-bit vs. 16-bit samples? Do I have to know that and cast the data appropriately, or can I force conversion to expected format?

lucasw

committed

16c637d

I thought I was getting good decoded data from the microphone, but now I only see five good samples and everything else is zero.

lucasw

committed

ad36501

Now seeing reasonable waveform, and it is 16-bit, which the viewer has to handle

lucasw

committed

e65a238

Passing data as -1.0 to 1.0 float now, so receiving node doesn't have to understand it (should be 0 to 1.0?)

lucasw

committed

3c51d1e

More visually pleasing but perhaps not super useful fade out of old waveforms

lucasw

committed

c87d14f

Generating a float array signal, now need to get it into gst encoded AudioData.

lucasw

committed

9095bed

Trying to convert ros messages of floating point arrays to gstreamer format, currently getting

lucasw

committed

094a920

Should be able to trace problem with debug on:

lucasw

committed

88f69cc

Trying out paused, but the next problem seems to be with pads? gstpad.c gst_pad_push_data error pushing events, return not-negotiated (Seen on debug level 6)

lucasw

committed

4677fbe

Trying out S16LE as the input type, but now back to 0:00:00.276624022 16823 0x224a000 WARN basetransform gstbasetransform.c:1414:gst_base_transform_setcaps:<filter> transform could n…

lucasw

committed

9e7d9a1

Trying out audio resample, no success yet

lucasw

committed

6a0592b

Finall got it! The whole problem was not specifying the layout as a string. The input and output rates do look different though

lucasw

committed

c8c1653

Making messages debug level, also adjusting visualization. Next step is to approximate nes sound chip with A-D voices, have four ros nodes in control of each, and then they can be played individua…

lucasw

committed

4fe1db6

Trying out nes style noise, but the waveform doesn't sound right

lucasw

committed

bba66bd

Wave encoding doesn't work, trying to fix that- if I put too many caps in it results in internal data error. Also trying to make noise sound more like online examples- I think when sample rate exc…

lucasw

committed

c642cbd

flac seems to work, though I got a crash once (maybe a different node though), and can't get rqt to load the config file

lucasw

committed

18b4d29

disabling wave. Flac works but the waveform is really different, it seems like there ought to be an auditory difference (or maybe I need the conversion to float to understand the decoded format be…

lucasw

committed

a4c7f70

Rename projects for consistency, but maybe float_to_audio should be audio_from_float to have the same prefix as the rest of the projects? Also deleted a lot of commented out code. Inability to ki…

lucasw

committed

52fe48c

The launch file doesn't work when I launch it- something is wrong with float_to_audio, but when I run it manually it does work:

lucasw

committed

2fc59f5

Forgot to add this headers

lucasw

committed

5b7fb73

Add options to disable parts of the float to audio launch file

lucasw

committed

4584078

View a spectrogram of the audio

lucasw

committed

8bae90d

Bigger nperseg shows more detail in the spectrogram

lucasw

committed

f6a80dc

flake8 compliance

lucasw

committed

497bba7

flake8 compliance for view.py

lucasw

committed

d61ee03

flake8 compliance in float_to_audio scripts

lucasw

committed

e0a3be8

Navigation Menu

Uh oh!

Convert AudioData stream to arrays of floats, and convert arrays of floats to AudioData - #84

Convert AudioData stream to arrays of floats, and convert arrays of floats to AudioData#84
lucasw wants to merge 31 commits into
ros-drivers:masterfrom
lucasw:appsink

Commits on May 19, 2023

x-raw allows my mic to work- needs to be parameter

Not compiling yet, want to be able to decode audio and then process arrays of samples.

py example doesn't work

This compiles but results in glib/gstreamer errors when it runs.

It looks like only one buffer is received, but I'm not handling it correctly so the callback gets triggered repeatedly with the same buffer. It seems like I need something else in the pipeline to …

This may be the raw data- but what about 8-bit vs. 16-bit samples? Do I have to know that and cast the data appropriately, or can I force conversion to expected format?

I thought I was getting good decoded data from the microphone, but now I only see five good samples and everything else is zero.

Now seeing reasonable waveform, and it is 16-bit, which the viewer has to handle

Passing data as -1.0 to 1.0 float now, so receiving node doesn't have to understand it (should be 0 to 1.0?)

More visually pleasing but perhaps not super useful fade out of old waveforms

Generating a float array signal, now need to get it into gst encoded AudioData.

Trying to convert ros messages of floating point arrays to gstreamer format, currently getting

Should be able to trace problem with debug on:

Trying out paused, but the next problem seems to be with pads? gstpad.c gst_pad_push_data error pushing events, return not-negotiated (Seen on debug level 6)

Trying out S16LE as the input type, but now back to 0:00:00.276624022 16823 0x224a000 WARN basetransform gstbasetransform.c:1414:gst_base_transform_setcaps:<filter> transform could n…

Trying out audio resample, no success yet

Finall got it! The whole problem was not specifying the layout as a string. The input and output rates do look different though

Making messages debug level, also adjusting visualization. Next step is to approximate nes sound chip with A-D voices, have four ros nodes in control of each, and then they can be played individua…

Trying out nes style noise, but the waveform doesn't sound right

Wave encoding doesn't work, trying to fix that- if I put too many caps in it results in internal data error. Also trying to make noise sound more like online examples- I think when sample rate exc…

flac seems to work, though I got a crash once (maybe a different node though), and can't get rqt to load the config file

disabling wave. Flac works but the waveform is really different, it seems like there ought to be an auditory difference (or maybe I need the conversion to float to understand the decoded format be…

Rename projects for consistency, but maybe float_to_audio should be audio_from_float to have the same prefix as the rest of the projects? Also deleted a lot of commented out code. Inability to ki…

The launch file doesn't work when I launch it- something is wrong with float_to_audio, but when I run it manually it does work:

Forgot to add this headers

Add options to disable parts of the float to audio launch file

View a spectrogram of the audio

Bigger nperseg shows more detail in the spectrogram

flake8 compliance

flake8 compliance for view.py

flake8 compliance in float_to_audio scripts