# Media interface

Interface for media data (audio, video and appsharing).

Create Media Channels.

Jitter Buffer.

Send and receive media data. (RTP/SCTP)

Protect media data. (Encryption/decryption, SRTP/DTLS)

Handling media devices. (Webcams, Microphones, Headsets)

## File information

File | common/interface/media.h
Classes | IMedia UMedia IMediaEndpoint UMediaEndpoint IMediaIoChannel UMediaIoChannel IDeviceIo UDeviceIo IAudioIoChannel IAudioIo UAudioIo IVideoIoChannel IVideoIo UVideoIo IAppSharingIoChannel IAppSharingIo UAppSharingIo IJitterBuffer ISrtpSession IMediaContainer UMediaContainer IWebcam UWebcam IWebcamProvider UWebcamProvider IMediaEncoder UMediaEncoder IMediaDecoder UMediaDecoder

# Classes

## IMedia

    class IMedia {
    public:
        virtual ~IMedia() {};
        virtual void Initialize(ISocketProvider * udpSocketProvider, ISocketProvider * tcpSocketProvider, class ISocketContext * socketContext, byte * certificateFingerprint, word minPort, word maxPort, const char * stunServers, const char * turnServers, const char * turnUsername, const char * turnPassword, enum MediaType media, bool stunSlow, bool turnOnly) = 0;
        virtual void Connect(class MediaConfig *remoteMediaConfig, bool iceControlling) = 0;
        virtual void RtpSend(const void * buf, size_t len, dword timestamp) = 0;
        virtual void SctpSend(const void * buf, size_t len) = 0;
        virtual void Recv(void * buf, size_t len, bool recvPartial = false) = 0;
        virtual void Close() = 0;
    };

### Overview

Media interface for connecting ICE/DTLS and send/receive media data.

## UMedia

    class UMedia {
    public:
        virtual void MediaInitializeComplete(IMedia * const media, class MediaConfig *localMediaConfig) {};
        virtual void MediaConnectResult(IMedia * const media, bool error) {};
        virtual void MediaRtpSendResult(IMedia * const media) {};
        virtual void MediaSctpSendResult(IMedia * const media) {};
        virtual void MediaRtpRecvResult(IMedia * const media, void * buf, size_t len, dword timestamp, bool sequenceGap) {};
        virtual void MediaSctpRecvResult(IMedia * const media, void * buf, size_t len) {};
        virtual void MediaCloseComplete(IMedia * const media) {};
        virtual void MediaEventReceived(IMedia * const media, enum MediaEndpointEvent event) {};
    };

### Overview

A class that receives updates from the media interface.

## IMediaEndpoint

    class IMediaEndpoint {
    public:
        virtual ~IMediaEndpoint() {};
        virtual void Recv(char * buf, int len, dword timestamp, short sequenceNumberDiff, bool marker) = 0;
        virtual void Send(char * buf, int len, dword timestamp) = 0;
    };

### Overview

A class that will receive the media data.

## UMediaEndpoint

    class UMediaEndpoint {
    public:
        virtual void RtpRecvResult(char * buf, int len, dword timestamp) {};
        virtual void RtpSendResult() {};
        virtual void RtpSend(char * buf, int len, dword timestamp) = 0;
        virtual void RtcpSend(char * buf, int len) = 0;
        virtual void SctpRecvResult(char * buf, int len) {};
        virtual void SctpSendResult() {};
        virtual void SctpSend(char * buf, int len) = 0;
    };

### Overview

A class that will receive updates from media endpoint.

## IMediaIoChannel

    class IMediaIoChannel {
    public:
        virtual ~IMediaIoChannel() {};
        virtual void Open() = 0;
        virtual void RtpRecv(void * buf, size_t len, dword timestamp) = 0;
        virtual void RtpSendResult() {};
        virtual void SctpRecv(void * buf, size_t len) = 0;
        virtual void SctpSendResult() {};
        virtual void Close() = 0;
        virtual void EventRecv(enum MediaEndpointEvent event) = 0;
    };

### Overview

Base class which defines a media channel.

## UMediaIoChannel

    class UMediaIoChannel {
    public:
        virtual void MediaIoRtpSend(const void * buf, size_t len, dword timestamp) = 0;
        virtual void MediaIoSctpSend(const void * buf, size_t len) = 0;
        virtual void MediaIoCloseComplete(class IMediaIoChannel * const mediaIoChannel) = 0;
    };

### Overview

A class that will receive updates from media channel.

## IDeviceIo

    class IDeviceIo {
    public:
        virtual void QueryDevices(void * context) = 0; // Calls DeviceAdded() for all present devices.
        virtual const char * StartDevice(void * context, void * src, const char *deviceId, int deviceMode) = 0;
        virtual void StopDevice(void * context, const char *deviceId) = 0;
    };

### Overview

A class for handling i/o devices.

## UDeviceIo

    class UDeviceIo {
    public:
        virtual void MediaIoDeviceAdded(void * context, const char *deviceId, int deviceType, unsigned deviceCapabilities, const char *deviceName) = 0;
        virtual void MediaIoDeviceRemoved(void * context, const char *deviceId) = 0;
    };

### Overview

A class that will receive updates from device interface.

## IAudioIoChannel

    class IAudioIoChannel : public IMediaIoChannel {
    public:
        static class IAudioIoChannel * Create(class IAudioIo * audioIo, class UMediaIoChannel * const user);
        static unsigned AvailableCoderCount();
        static enum AudioCoder AvailableCoder(unsigned coderNumber);
        virtual ~IAudioIoChannel() {};
        virtual void Initialize(enum AudioCoder coder, bool sc, unsigned mediaPacketizationMs, unsigned execInterval8khz,
            unsigned execJitter8khz, unsigned pullSampleRate, unsigned putSampleRate) = 0;
        virtual bool Pull() = 0;
        virtual const short *PulledSamples() = 0;
        virtual const short *FeedbackSamples() = 0;
        virtual void PutSamples(const short *buffer) = 0;
    };

### Overview

A class that handles an audio channel, derived from IMediaIoChannel.

## IAudioIo

    class IAudioIo : public IDeviceIo {
    public:
        static class IAudioIo * Create(class IIoMux * const iomux, class IAudioPhone * const audioPhone, unsigned execInterval8khz);
        virtual ~IAudioIo() {};
        virtual class IAudioPhone *Initialize(class UAudioIo * const user, class UDeviceIo * const deviceIoUser) = 0;
        virtual void AudioExec(class IAudioExec *audioExec) = 0;
        virtual void StartMonitoring(const char *deviceId, int deviceMode) = 0;
        virtual void StopMonitoring(const char *deviceId) = 0;
        virtual void StartHookDevice(const char *deviceId) = 0;
        virtual void StopHookDevice(const char *deviceId) = 0;
        virtual void SendHookKey(const char *deviceId, byte key) = 0;
        virtual void StartDualTones(dword toneFlags, unsigned toneCount, const struct AudioIoDualTone *tones) = 0;
        virtual void StopDualTones() = 0;
        virtual class IAudioIoChannel *CreateChannel(class UMediaIoChannel * const user) = 0;
        virtual void InitializeChannel(class IAudioIoChannel *audioIoChannel, enum AudioCoder coder, bool sc, unsigned mediaPacketizationMs) = 0;
        virtual void ChannelOpened(class IAudioIoChannel *audioIoChannel) = 0;
        virtual void ChannelClosed(class IAudioIoChannel *audioIoChannel) = 0;
        virtual void ChannelDestroyed(class IAudioIoChannel *audioIoChannel) = 0;
        virtual void Close(class UAudioIo * user) = 0;
    };

### Overview

Interface that will handle audio i/o devices, derived from IDeviceIo.

## UAudioIo

    class UAudioIo {
    public:
        virtual void HookDeviceMsg(const char* deviceId, const char* msg) = 0;
        virtual void HookKeyReceived(const char *deviceId, byte key) = 0;
        virtual void CloseAudioIoComplete() = 0;
    };

### Overview

Interface that will receive updates from the audio interface.

## IVideoIoChannel

    class IVideoIoChannel : public IMediaIoChannel {
    public:
        static class IVideoIoChannel * Create(class IIoMux * const iomux, class IVideoIo * videoIo, class UMediaIoChannel * const user, class IInstanceLog * log);
        static unsigned AvailableCoderCount();
        static enum VideoCoder AvailableCoder(unsigned coderNumber);
        virtual ~IVideoIoChannel() {};
        virtual void Initialize(void * context, const char * channelId, enum VideoCoder coder, ConnectionType connType) = 0;
    };

### Overview

A class that handles a video channel, derived from IMediaIoChannel.

## IVideoIo

    class IVideoIo : public IDeviceIo {
    public:
        static class IVideoIo * Create(class IIoMux * const iomux, class IInstanceLog * log);
        virtual ~IVideoIo() {};
        virtual void Initialize(class UVideoIo * const user, class UDeviceIo * const deviceIoUser) = 0;
        virtual void RemoveLocalContainer(void * container, enum MediaType coder) = 0;
        virtual void AddRemoteContainer(void * context, void * container, const char * channelId, enum MediaType coder) = 0;
        virtual void RemoveRemoteContainer(void * container, enum MediaType coder) = 0;
        virtual void StartVideoEncoder(void * context, const char * channelId, int codec) = 0;
        virtual void Close() = 0;
    };

### Overview

Interface that will handle video i/o devices, derived from IDeviceIo.

## UVideoIo

    class UVideoIo {
    public:
        virtual void CloseVideoIoComplete() = 0;
    };

### Overview

Interface that will receive updates from the video i/o interface.

## IAppSharingIoChannel

    class IAppSharingIoChannel : public IMediaIoChannel {
    public:
        static class IAppSharingIoChannel * Create(class IIoMux * const iomux, class IAppSharingIo * appSharingIo, class UMediaIoChannel * const user, class IInstanceLog * log, const char * dn);
        static unsigned AvailableCoderCount();
        static enum AppSharingCoder AvailableCoder(unsigned coderNumber);
        virtual ~IAppSharingIoChannel() {};
        virtual void Initialize(void * context, const char * channelId, enum AppSharingCoder coder) = 0;
    };

### Overview

A class that handles a appsharing channel, derived from IMediaIoChannel.

## IAppSharingIo

    class IAppSharingIo : public IDeviceIo {
    public:
        static class IAppSharingIo * Create(class IIoMux * const iomux, class IInstanceLog * log);
        virtual ~IAppSharingIo() {};
        virtual void Initialize(class UAppSharingIo * const user, class UDeviceIo * const deviceIoUser) = 0;
        virtual void SignalingMessage(void * context, void * buffer, int len) = 0;
        virtual void SubscribeApplications() = 0;
        virtual void UnsubscribeApplications() = 0;
        virtual void ShareApplication(unsigned int id) = 0;
        virtual void UnshareApplication(unsigned int id) = 0;
        virtual void UnshareAllApplications() = 0;
        virtual void GiveControlToUser(unsigned int userId) = 0;
        virtual void RemoveControlFromUser(unsigned int userId) = 0;
        virtual void AddRemoteContainer(void * context, void * container, const char * channelId, enum MediaType coder) = 0;
        virtual void RemoveRemoteContainer(void * container, enum MediaType coder) = 0;
        virtual void Close() = 0;
    };

### Overview

Interface that will handle appsharing i/o devices, derived from IDeviceIo.

## UAppSharingIo

    class UAppSharingIo {
    public:
        virtual void AddAppSharingApplication(unsigned int id, const char * name, const char * desc, const char * icon,  const char * thumbnail) = 0;
        virtual void RemoveAppSharingApplication(unsigned int id) = 0;
        virtual void UpdateAppSharingApplication(unsigned int id, const char * name, const char * desc, const char * icon,  const char * thumbnail) = 0;
        virtual void CloseAppSharingIoComplete() = 0;
        virtual void MediaIoAppSharingContainerSend(const void * buf, size_t len, dword timestamp, const void * context, bool local, enum MediaType type) = 0;
        virtual void RequestAppSharingControl(unsigned int userId) = 0;
    };

### Overview

Interface that will receive updates from the appsharing i/o interface.

## IJitterBuffer

    class IJitterBuffer {
    public:
        static IJitterBuffer * Create(class UJitterBuffer * user);
        virtual ~IJitterBuffer() {};
        virtual unsigned Configure(enum AudioCoder coder, unsigned mediaPacketizationMs, unsigned pullInterval8khz,
            unsigned minBufferMs, unsigned initialBufferMs, unsigned maxBufferMs) = 0;
        virtual void Adjust(unsigned initialBufferMs) = 0;
        virtual unsigned GetDelay() = 0;
        virtual void Push(const void * buf, size_t len, dword timestamp) = 0;
        virtual bool Pull(bool discard) = 0;
    };

### Overview

Base class for a jitter buffer.

## UJitterBuffer

    class UJitterBuffer {
    public:
        virtual void JitterBufferPulled(IJitterBuffer * const jitterBuffer, const void * buf, size_t len) = 0;
    };

### Overview

Interface that will receive updates from the jitterbuffer interface.

## ISrtpSession

    class ISrtpSession {
    public:
        static ISrtpSession * Create(byte * srtpMasterkey, byte * srtpMastersalt, const char * profile);
        virtual ~ISrtpSession() {};
        virtual byte * Protect(byte * header, int hLen, byte * payload, int pLen, int * olen) = 0;
        virtual int Unprotect(char * srtpPacket, int len) = 0;
        virtual byte * SrtcpProtect(byte * rtcpPacket, int pLen, int * olen) = 0;
        virtual int SrtcpUnprotect(char * srtcpPacket, int len) = 0;
    };

### Overview

Interface that will protect and unprotect media data.

## IMediaContainer

    class IMediaContainer {
    public:
        virtual ~IMediaContainer() {};
        virtual void PutSample(char * buf, int len, dword timestamp) = 0;
        virtual void Initialize(void) = 0;
    };

### Overview

Interface that will define a media container (webm, mp4).

## UMediaContainer

    class UMediaContainer {
    public:
        virtual void ContainerSample(char * buf, int len) = 0;
    };

### Overview

Interface that will receive updates from the IMediaContainer interface.

## IWebcam

    class IWebcam {
    public:
        virtual ~IWebcam() {};
        virtual struct VideoFrameFormat * Start() = 0;
        virtual void Stop() = 0;
        virtual void Close() = 0;
        virtual const char * GetWebcamId() = 0;
        virtual const char * GetWebcamName() = 0;
        virtual void FullIntraRequest() {};
    };

### Overview

Interface that will define a webcam.

## UWebcam

    class UWebcam {
    public:
        virtual void WebcamVideoFormat(class IWebcam * const webcam, const struct VideoFrameFormat * format) = 0;
        virtual void WebcamSample(class IWebcam * const webcam, const void * buf, int len, dword timestamp, dword duration, const struct VideoFrameFormat * format) = 0;
    };

### Overview

Interface that will receive updates from the IWebcam interface.

## IWebcamProvider

    class IWebcamProvider {
    public:
        static class IWebcamProvider * Create(class IIoMux * const iomux, class UWebcamProvider * const webcamProviderUser, class UWebcam * const webcamUser, class IInstanceLog * const log);
        virtual ~IWebcamProvider() {};
        virtual void QueryDevices(void * context) = 0;
        virtual void Close() = 0;
    };

### Overview

Interface that implements a webcam provider.

## UWebcamProvider

    class UWebcamProvider {
    public:
        virtual void WebcamLost(class IWebcam * const webcam, void * context) = 0;
        virtual void WebcamAdded(class IWebcam * const webcam, void * context) = 0;
        virtual void WebcamClosed(class IWebcam * const webcam) = 0;
        virtual void WebcamProviderCloseResult() = 0;
    };

### Overview

Interface that will receive updates from the webcam provider.

## IMediaEncoder

    class IMediaEncoder {
    public:
        static class IMediaEncoder * Create(class IIoMux * const iomux, class UMediaEncoder * const user, class IInstanceLog * log, enum VideoCoder coder);
        virtual ~IMediaEncoder() {};
        virtual bool Initialize(struct VideoFrameFormat * const format) = 0;
        virtual void Encode(const void * buf, int len, dword timestamp, dword duration, const struct VideoFrameFormat * f) = 0;
        virtual bool FullIntraRequest() = 0;
        virtual void Close() = 0;
    };

### Overview

Interface that implements a media encoder.

## UMediaEncoder

    class UMediaEncoder {
    public:
        virtual void EncodeResult(class IMediaEncoder * encoder, const void * buf, int len, dword timestamp, dword duration) = 0;
        virtual void MediaEncoderCloseResult(class IMediaEncoder * encoder) = 0;
    };

### Overview

Interface that will receive updates from the media encoder.

## IMediaDecoder

    class IMediaDecoder {
    public:
        static class IMediaDecoder * Create(class IIoMux * const iomux, class UMediaDecoder * const user, class IInstanceLog * log, enum VideoCoder coder);
        virtual ~IMediaDecoder() {};
        virtual void Decode(const void *buf, int len, dword timestamp) = 0;
        virtual void Close() = 0;
    };

### Overview

Interface that implements a media decoder.

## UMediaDecoder

    class UMediaDecoder {
    public:
        virtual void DecodeResult(class IMediaDecoder * decoder, const void * buf, int len, dword timestamp, dword duration, struct VideoFrameFormat * f) = 0;
        virtual void MediaDecoderCloseResult(class IMediaDecoder * decoder) = 0;
    };

### Overview

Interface that will receive updates from the media decoder.