sounds in bada

Sounds in bada

Copyright © 2010 Samsung Electronics Co., Ltd. All rights reserved. 1

Contents
Introduction
Common Sound Classes
– AudioRecorder/Player
– AudioIn/AudioOut
Simple Mixing
Media Management
– Image
– Video
– Streaming
Summary
*This material is based on bada SDK 1.0.0b3

Media Introduction
Audio/ video/ image processing functions
Key features

Audio Video Image
• Playback • Playback • Load
• Record • Record • Save
• Streaming • Streaming • Format conversion
• PCM processing • Camera capture
• Camera preview


Supported Formats
Type Format Encode Decode Streaming*
Video H.263, MPEG-4 (.3gp, mp4) O O O
H.264 (.3gp, mp4) O O
VC-1 (.wmv, .asf) O
Audio PCM (.wav) O O
AMR-NB (.amr, .3ga, .m4a) O O O
AAC, AAC+, EAAC+ O O
(.aac, .3ga, .m4a)
MIDI (.mid, .spm, .imy, .mmf, .xmf) O
MP3 (.mp3), WMA ( .wma, .asf) O
Image JPEG, PNG, BMP O O
GIF, TIFF, WBMP O

*Streaming Protocol: RTSP

Header & Library
Header file:
– #include <FMedia.h>
Library
– Simulator: FMedia
– Target: FOsp
Privilege groups
– RECORDING
– IMAGE
– CAMERA

Common Sound Classes


Audio Recorder & Player
Basic method Complex method
:General purpose :Direct access to device

Recording Recording
– AudioRecorder : Records – AudioIn : Read raw PCM
audio data from input data from input srouce
source to storage
Playing Playing
– Player : Open audio file – AudioOut : Play sound with
and play raw PCM data

*PCM: Pulse Code Modulation Copyright © 2010 Samsung Electronics Co., Ltd. All rights reserved. 7

Demo
Playback
– Player
– AudioOut


Basic method


AudioRecorder class
Records audio data from input source to storage
– Recording format
• WAV, AMR
Event
– IAudioRecorderEventListener
• OnAudioRecorderPaused
• OnAudioRecorderStarted
• OnAudioRecorderStopped
• OnAudioRecorderClosed
• OnAudioRecorderCanceled
• …
"RECORDING" privilege is needed

Using the AudioRecorder class
Create AudioRecorder instance
__pAR = new AudioRecorder;
__pAR->Construct(*pListener);
// IAudioRecorderEventListener

Recording
__pAR->CreateAudioFile( __path, true);
__pAR->SetFormat(AUDIORECORDING_FORMAT_WAVE);
__pAR->Record();
Stop recording & close
__pAR->Stop();
…
void EventListener::OnAudioRecorderStopped( result r)
{
__pAR->Close();
}

Player class
Can play audio & video
– Multiple audio playback is supported(max 10)
• Only 1 compressed audio files
– Can play audio/video by streaming
• Streaming Protocol: Real-Time Streaming Protocol
(RTSP)
Event
– IPlayerEventListener
• OnPlayerBuffering, OnPlayerInterrupted
• OnPlayerEndOfClip, OnPlayerErrorOccurred
• OnPlayerOpened, OnPlayerReleased
• OnPlayerSeekCompleted

Using the Player class
Create Player instance
__pPlayer = new Player;
__pPlayer->Construct(*pListener);
// IPlayerEventListener

Load file & play
result r = __pPlayer->OpenFile(__path);
if (IsFailed(r)) return;
__pPlayer->Play();

Stop
__pPlayer->Stop();
__pPlayer->Close();


Complex method


Record PCM data
AudioIn class
– Record raw uncompressed PCM data from
microphone
– Double buffering is possible
Event Listener
– IAudioInEventListener
• OnAudioInBufferIsFilled
• OnAudioInInterrupted
• OnAudioInReleased


Using the AudioIn class
Create AudioIn instance
__pAudioIn = new AudioIn();
__pAudioIn->Construct(*__pListener);
__pAudioIn-> Prepare (AUDIO_INPUT_DEVICE_MIC,
AUDIO_TYPE_PCM_U8, AUDIO_CHANNEL_TYPE_STEREO,8000);

Feed buffers and start recording
__pAudioIn->AddBuffer(__pByteBuffer1);
__pAudioIn->Start();

Feed next buffers
void Listener::OnAudioInBufferIsFilled
( Osp::Base::ByteBuffer* pData)
{ // save result.
pData->Clear();
__pAudioIn->AddBuffer(pData);

Play PCM data
AudioOut
– Play raw uncompressed PCM data
– Double buffering is possible
Event listener
– IAudioOutEventListener
• OnAudioOutBufferEndReached
• OnAudioOutInterrupted
• OnAudioOutReleased


Using the AudioOut class
Create AudioOut instance
__pAO = new AudioOut();
__pAO->Construct(*pListener);// IAudioOutEventListener
__pAO->Prepare(AUDIO_TYPE_PCM_S16_LE,
AUDIO_CHANNEL_TYPE_MONO, SAMPLERATE);
__pOutBuffer = new ByteBuffer();
__pOutBuffer->Construct(BUFSIZE);

Feed PCM data
// Fill buffer
__pAO->WriteBuffer(*__pOutBuffer);

Play sound
__pAO->Start();


Using the AudioOut class ctd.
Process events
void EventListener::OnAudioOutBufferEndReached
(Osp::Media::AudioOut &src)
{ //Feed Next PCM data }

Stop sound
__pAO->Stop(); //or Reset()


Double buffering in AudioIn
Step 1. Initialize AudioIn Recording
AudioIn Device
application


AudioIn Device
Step 2. Add two buffers application

Buffer1

Buffer2


AudioIn Device

Step 3. Start recording
Step 4. Buffer1 is filled

Buffer1

Buffer2


AudioIn Device

Step 3. Start recording Event:
Buffer is filled
Step 5. Event is fired Buffer1
Buffer2 is used by AudioIn
Buffer2


AudioIn Device

Buffer2
Step 6. App. reads Buffer1
and AudioIn writes to Buffer2


AudioIn Device

Step 3. Start recording Event:
Buffer is filled
Buffer2
Step 7. Event is fired
AudioIn switches to Buffer1


AudioIn Device

Buffer2
AudioIn switches to Buffer1
Step 8. Read Buffer2


Double buffering in AudioOut
Step 1. Initialize AudioOut Player
AudioOut Device
application


AudioOut Device
application
Step 2. Add two buffers

Buffer1

Buffer2


AudioOut Device
application
Step 3. Start playback
Step 4. Buffer1 is read

Buffer1

Buffer2


AudioOut Device
application
Event:
Buffer end is reached
Buffer2 is used by AudioOut
Buffer2


AudioOut Device
application
Buffer2
Step 6. AudioOut reads Buffer2
and app writes to Buffer1


AudioOut Device
application
Event:
Buffer end is reached
Buffer2
AudioOut switches to Buffer1


AudioOut Device
application
Buffer2
AudioOut switches to Buffer1
Step 8. Write to Buffer2


Simple Mixing


Why mixing?
Play many sounds simultaneously
– Limited number of player instances
– System resource limited
Performance issue
– Consume less computing power
Flexibility
– Finer control of PCM data is possible


.wav file
Field Field
Offset Field Name Size
(bytes) (bytes)
Important .wav header 0
Chunk ID 4
4
RIFFHeader
fields 8
Chuck Size
Format
4
4
12
Subchunk1 ID 4
16
Field Value Subchunk1 Size 4
20
Audio Format 2
Channels mono/stereo 22
Num Channels 2
Sample rate 8/22/44.1Khz 24
Sample Rate 4
WavFormatInfo
28
Byte Rate 4
Bit per sample 8bit /16bit 32
Block Align 2
Size raw data size 34
Bits Per Sample 2
36
Subchunk2 ID 4
PCM data raw data 40 ChnkHeader
Subchunk2 Size 4
44

Subchunk2size
Data


Loading .wav file
Open file
File f;
f.Construct(/*File Path*/, String(L"r"));

Read header
f.Read(&__RIFFHeader, sizeof(RIFFHEader));
f.Read(&__WavFormat, sizeof(WavFormatInfo));

Read PCM data
ChnkHeader tmp;
f.Read((void *)&tmp, sizeof(ChnkHeader));
__pData = new Osp::Base::ByteBuffer;
__pData->Construct(tmp.size);
f.Read(*__pData);
__pData->Rewind();


Simple mixing
Step 1: Create buffers
__pTmpBuf = new ByteBuffer();
__pTmpBuf->Construct(size*4); //int type

Step 2: Add the buffers
int* pTmp =
static_cast<int *> (__pTmpBuf->GetPointer());
short* pData1 =
static_cast<short *> (__pData1->GetPointer());
short* pData2 =
static_cast<short *> (__pData2->GetPointer());
for (int i = 0; i < size; i++)
pTmp[i] = pData1[i] + pData2[i] ;


Simple mixing
Step 3: Clip the buffers
short * pOut =
static_cast<short *> (__pOutBuf->GetPointer());

for( int i=0;i<size;i++)
{
if (pTmp[i] >= SHRT_MAX) pOut[i] = SHRT_MAX*;
else if ( pTmp[i] <= SHRT_MIN) pOut[i] = SHRT_MIN*;
else pOut[i] = pTmp[i];
}
*SHRT_MAX, SHRT_MIN: maximum and minimum values of short type

Step 4: Write the buffer to AudioOut
__pAO->WriteBuffer(*__pOutBuf);


Image & Video management
• Image
• Video
• Streaming


Image
Can encode, decode, convert images
Supported pixel format
– RGB565/ARGB8888/R8G8B8A8
Decoding
– BMP, GIF, PNG, TIFF, WBMP
Encoding
– BMP, JPEG, PNG
"IMAGE" privilege is needed


Image class decode/encode
Load image
Bitmap* LoadBitmapN(const String& name)
{
Bitmap* pBitmap = null;
Image* pImage = new Image();
pImage->Construct();
pBitmap = pImage->DecodeN(name,
BITMAP_PIXEL_FORMAT_ARGB8888);
delete pImage;
return pBitmap;
}

Save image
pImage->EncodeToFile(*pBitmap,
IMG_FORMAT_PNG, name, true);


Video playing
Can play video using Player class
– Using OverlayPanel to display video
Code snippet
– Create player
pPlayer = new Player();
pPlayer->Construct(*pListener, &bufferInfo);
– Open file
pPlayer->OpenFile(String(L"/Res/badaBI.mp4"));
– Play/stop/pause/close
pPlayer->Play(); //Pause() or Stop() or Close()


OverlayPanel
Primarily used for the playback of camera
previews or video
– Foreground panel
• Overlay graphics and controls
– Background buffer
• Support H/W accelerated rendering
Foreground panel

Input buffer
Masking color

Background buffer Copyright © 2010 Samsung Electronics Co., Ltd. All rights reserved. 44

OverlayPanel ctd.
Create a panel and get bufferInfo
pPanel= new OverlayPanel();
pPanel->Construct(rect);
AddControl(*pPanel);
pPanel->GetBackgroundBufferInfo(bufferInfo);

Use the bufferInfo in the Player class
pPlayer->Construct(*pListener, &bufferInfo);


Streaming
Can play audio/video streaming content
– Supported protocol : RTSP
– Supported video codecs : MPEG4, H.263,
H.264
– Supported audio codecs : AAC, HE-AAC,
Enhanced-AAC+, AMR-NB
pPlayer->OpenURL(
String(L“RTSP://IP address/content"));


Demo
MediaPlayer
– Video Playback
– OverlayPanel


Tips for Performance
Preffered PCM format to reduce internal
operations
Field Values
Channels stereo
Sample rate 44.1Khz
Bit per sample 16bit

Double buffering to reduce latency
Sound mixing can reduce internal resources and
event processing


Tips for Programming
Handled automatically by platform
Headset • Plug and unplug events are not fired
• Platform changes the sound path automatically
Incoming • Player, AudioIn, and AudioOut are stopped by
call platform
• Resuming should be done by application
(Osp::App::Application::OnForeground)


Tips for Programming
Handled by application
Silent mode • Event is not fired
• Application should check silent mode
(Osp::System::SettingInfo )
Side volume • Application should deal with volume controls
key • Application handles key event
(Osp::Ui::IKeyEventListener)
Foreground/ • Player consumes too much power
background • On background, stop playing audio and
video


Summary


What we have learned
Basic way to record and play audio
– AudioRecorder / Player classes
Playing and recording PCM data
– AudioIn / AudioOut classes
– Wav file format / Simple mixing
Image and Video management
– Image encode/decode
– OverlayPanel for video
– Streaming both audio and video


Find out more
Tutorial
– bada Tutorial.Media.pdf
Samples
– MediaPlayer
– VoiceRecorder


Dive into
http://www.goprodiver.com

sounds in bada

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (20)

Similaire à sounds in bada

Similaire à sounds in bada (20)

Plus de Samsung

Plus de Samsung (7)

Dernier

Dernier (20)

sounds in bada