how to record video in one device at that same time play that video live in another device

Real-time Video Streaming: Recording on One Device, Playing on Another

Learn how to set up a system for recording video on an Android device and simultaneously streaming it live to another device, covering key concepts and implementation strategies.

The ability to record video on one device and instantly stream it to another opens up a wide range of applications, from security monitoring and remote assistance to live event broadcasting and collaborative content creation. This article will guide you through the fundamental concepts and technical approaches required to achieve real-time video streaming between two Android devices. We'll explore the core components involved, including video capture, encoding, network transmission, and playback, providing a solid foundation for building your own live streaming solution.

Understanding the Core Components

Achieving real-time video streaming involves several critical steps that must be orchestrated seamlessly. These steps include capturing raw video frames from the camera, encoding them into a compressed format, transmitting the encoded data over a network, and finally, decoding and rendering the video on the receiving device. Each component plays a vital role in ensuring low-latency and high-quality streaming.

graph TD
    A[Recording Device] --> B{Video Capture (Camera)};
    B --> C{Video Encoding (H.264/VP8)};
    C --> D{Network Transmission (RTSP/WebRTC)};
    D --> E[Receiving Device];
    E --> F{Network Reception};
    F --> G{Video Decoding};
    G --> H{Video Playback (SurfaceView)};

High-level architecture for real-time video streaming between two devices.

Video Capture

On the recording device, the first step is to access the camera and capture video frames. Android's CameraX or the older Camera2 API are the primary tools for this. CameraX offers a simpler, more consistent API across various Android versions, making it a preferred choice for new development. It provides use cases like Preview, ImageCapture, and VideoCapture.

Video Encoding

Raw video frames are uncompressed and very large, making them unsuitable for network transmission. They must be encoded into a compressed format. Common video codecs include H.264 (AVC) and H.265 (HEVC) for high efficiency, and VP8/VP9 for web-friendly streaming. Android's MediaCodec API allows direct access to hardware video encoders, which are crucial for performance and battery life in real-time applications.

Network Transmission

Once encoded, the video data needs to be sent over a network. Several protocols can be used:

• RTSP (Real-Time Streaming Protocol): A common protocol for controlling streaming media servers. It's often paired with RTP (Real-time Transport Protocol) for the actual data transmission.
• WebRTC (Web Real-Time Communication): A powerful, open-source project that enables real-time communication (video, audio, and data) directly between browsers and mobile applications. It handles NAT traversal, firewall negotiation, and provides low-latency, peer-to-peer connections.
• Custom TCP/UDP Sockets: For highly specialized or low-level control, you can implement your own streaming protocol over raw TCP or UDP sockets, though this requires significant effort in error handling, retransmission, and congestion control.

Video Decoding and Playback

On the receiving device, the process is reversed. The incoming network stream is received, the encoded video data is extracted, decoded using MediaCodec, and then rendered onto a SurfaceView or TextureView for display. Low latency is paramount here, requiring efficient decoding and rendering pipelines.

Implementing Video Capture and Encoding (Recording Device)

Let's outline the basic steps for capturing video and preparing it for streaming using Android's CameraX and MediaCodec.

import android.Manifest
import android.content.pm.PackageManager
import android.os.Bundle
import android.util.Log
import android.view.Surface
import android.view.SurfaceHolder
import android.view.SurfaceView
import androidx.appcompat.app.AppCompatActivity
import androidx.camera.core.CameraSelector
import androidx.camera.core.Preview
import androidx.camera.lifecycle.ProcessCameraProvider
import androidx.camera.video.Quality
import androidx.camera.video.QualitySelector
import androidx.camera.video.Recorder
import androidx.camera.video.Recording
import androidx.camera.video.VideoCapture
import androidx.core.app.ActivityCompat
import androidx.core.content.ContextCompat
import java.util.concurrent.ExecutorService
import java.util.concurrent.Executors

class CameraActivity : AppCompatActivity() {

    private lateinit var cameraExecutor: ExecutorService
    private var videoCapture: VideoCapture<Recorder>? = null
    private var recording: Recording? = null
    private lateinit var previewView: SurfaceView // Or PreviewView from CameraX

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_camera)

        previewView = findViewById(R.id.preview_view)
        cameraExecutor = Executors.newSingleThreadExecutor()

        if (allPermissionsGranted()) {
            startCamera()
        } else {
            ActivityCompat.requestPermissions(
                this,
                REQUIRED_PERMISSIONS,
                REQUEST_CODE_PERMISSIONS
            )
        }
    }

    private fun startCamera() {
        val cameraProviderFuture = ProcessCameraProvider.getInstance(this)

        cameraProviderFuture.addListener({ 
            val cameraProvider: ProcessCameraProvider = cameraProviderFuture.get()

            val preview = Preview.Builder()
                .build()
                .also { it.setSurfaceProvider(previewView.surfaceHolder) }

            val recorder = Recorder.Builder()
                .setQualitySelector(QualitySelector.from(Quality.HD))
                .build()

            videoCapture = VideoCapture.withOutput(recorder)

            val cameraSelector = CameraSelector.DEFAULT_BACK_CAMERA

            try {
                cameraProvider.unbindAll()
                cameraProvider.bindToLifecycle(
                    this,
                    cameraSelector,
                    preview,
                    videoCapture
                )
            } catch (exc: Exception) {
                Log.e(TAG, "Use case binding failed", exc)
            }
        }, ContextCompat.getMainExecutor(this))
    }

    // This is where you would integrate MediaCodec for custom encoding
    // For live streaming, you'd typically get frames from an ImageAnalysis use case
    // or configure VideoCapture to output to a Surface that MediaCodec consumes.
    private fun setupMediaCodecEncoder(surface: Surface) {
        // Example: Configure MediaCodec for H.264 encoding
        // This part is complex and involves creating MediaFormat, MediaCodec.createEncoderByType,
        // configuring it, and feeding it raw frames or connecting it to a Surface.
        // For CameraX VideoCapture, you can directly provide a Surface from MediaCodec's createInputSurface().
        Log.d(TAG, "MediaCodec encoder setup would go here.")
    }

    private fun allPermissionsGranted() = REQUIRED_PERMISSIONS.all {
        ContextCompat.checkSelfPermission(baseContext, it) == PackageManager.PERMISSION_GRANTED
    }

    override fun onDestroy() {
        super.onDestroy()
        cameraExecutor.shutdown()
    }

    companion object {
        private const val TAG = "CameraXLiveStream"
        private const val REQUEST_CODE_PERMISSIONS = 10
        private val REQUIRED_PERMISSIONS = arrayOf(Manifest.permission.CAMERA, Manifest.permission.RECORD_AUDIO)
    }
}

Basic CameraX setup for video capture. For live streaming, VideoCapture can output to a Surface provided by MediaCodec's input surface.

Network Transmission and Playback (Receiving Device)

The choice of network protocol significantly impacts latency and complexity. WebRTC is generally preferred for low-latency, peer-to-peer scenarios, while RTSP/RTP might be used for server-based streaming or when integrating with existing IP camera systems.

sequenceDiagram
    participant Rec as Recording Device
    participant Net as Network
    participant Play as Playback Device

    Rec->>Net: Encoded Video Stream (e.g., WebRTC/RTSP)
    Net-->>Play: Encoded Video Stream
    Play->>Play: Decode Video (MediaCodec)
    Play->>Play: Render to SurfaceView
    Note over Play: Low-latency playback is crucial

Sequence diagram for network transmission and playback.

WebRTC for Low-Latency Streaming

WebRTC is an excellent choice for peer-to-peer live streaming due to its built-in support for low latency, NAT traversal, and security. It handles much of the complexity of real-time communication. Integrating WebRTC involves setting up PeerConnection objects on both devices, exchanging SDP (Session Description Protocol) offers/answers, and ICE (Interactive Connectivity Establishment) candidates to establish a connection.

RTSP/RTP for Server-Based or IP Camera Integration

If you need to stream to a central server or integrate with existing RTSP-compatible systems, implementing an RTSP server on the recording device (or using a library that provides one) and an RTSP client on the playback device is an option. This typically involves using libraries like libstreaming or building a custom solution on top of Socket APIs.

import android.media.MediaCodec
import android.media.MediaFormat
import android.view.SurfaceHolder
import android.view.SurfaceView
import java.nio.ByteBuffer

class VideoPlayer(private val surfaceView: SurfaceView) {

    private var mediaCodec: MediaCodec? = null
    private var isPlaying = false

    fun startPlayback(videoFormat: MediaFormat) {
        if (isPlaying) return

        surfaceView.holder.addCallback(object : SurfaceHolder.Callback {
            override fun surfaceCreated(holder: SurfaceHolder) {
                try {
                    mediaCodec = MediaCodec.createDecoderByType(videoFormat.getString(MediaFormat.KEY_MIME)!!)
                    mediaCodec?.configure(videoFormat, holder.surface, null, 0)
                    mediaCodec?.start()
                    isPlaying = true
                    // Start a thread to feed data to the decoder
                    // This is where you'd receive network data and feed it to the decoder
                    Thread { decodeLoop() }.start()
                } catch (e: Exception) {
                    e.printStackTrace()
                }
            }

            override fun surfaceChanged(holder: SurfaceHolder, format: Int, width: Int, height: Int) {}
            override fun surfaceDestroyed(holder: SurfaceHolder) {
                stopPlayback()
            }
        })
    }

    private fun decodeLoop() {
        val TIMEOUT_US = 10000L
        val bufferInfo = MediaCodec.BufferInfo()

        while (isPlaying) {
            // In a real scenario, you'd get encoded data from your network receiver
            val encodedData: ByteBuffer? = getEncodedDataFromNetwork()

            if (encodedData != null) {
                val inputBufferId = mediaCodec?.dequeueInputBuffer(TIMEOUT_US) ?: -1
                if (inputBufferId >= 0) {
                    val inputBuffer = mediaCodec?.getInputBuffer(inputBufferId)
                    inputBuffer?.clear()
                    inputBuffer?.put(encodedData)
                    mediaCodec?.queueInputBuffer(inputBufferId, 0, encodedData.limit(), System.nanoTime() / 1000, 0)
                }
            }

            val outputBufferId = mediaCodec?.dequeueOutputBuffer(bufferInfo, TIMEOUT_US) ?: -1
            if (outputBufferId >= 0) {
                mediaCodec?.releaseOutputBuffer(outputBufferId, true) // Render to surface
            }
        }
    }

    private fun getEncodedDataFromNetwork(): ByteBuffer? {
        // Placeholder: In a real application, this would read from your network stream
        // For example, from a WebRTC DataChannel or an RTSP client's buffer.
        return null // Return actual ByteBuffer with encoded frame data
    }

    fun stopPlayback() {
        isPlaying = false
        mediaCodec?.stop()
        mediaCodec?.release()
        mediaCodec = null
    }
}

Basic MediaCodec decoder setup for video playback on the receiving device.

Key Considerations for Real-time Performance

Achieving truly real-time, low-latency streaming requires attention to several factors beyond just the core components.

Latency Optimization

• Codec Choice: Hardware-accelerated codecs (H.264, H.265) are essential. Software codecs introduce significant latency.
• GOP Structure: Use a small Group of Pictures (GOP) size (e.g., 1-2 seconds) for encoding. A smaller GOP means more I-frames, which are independently decodable, reducing the need to wait for previous frames.
• Buffer Management: Minimize buffering on both the sender and receiver. Excessive buffering increases latency.
• Network Protocol: WebRTC is designed for low latency. If using custom sockets, implement efficient packetization and retransmission strategies.

Network Reliability

• UDP vs. TCP: UDP is often preferred for real-time media due to its lower overhead and lack of retransmission delays (which can cause stuttering). However, it requires application-level error handling. TCP guarantees delivery but can introduce latency due to retransmissions.
• Congestion Control: Implement mechanisms to adapt to changing network conditions, such as adjusting bitrate or frame rate.

Audio Synchronization

If streaming audio as well, ensure proper synchronization between video and audio streams. This typically involves timestamping frames and samples and using a common clock reference during playback.