io.cine.android

Class BroadcastActivity

java.lang.Object

android.content.Context

android.content.ContextWrapper

android.view.ContextThemeWrapper

android.app.Activity

io.cine.android.BroadcastActivity

All Implemented Interfaces:

android.content.ComponentCallbacks, android.content.ComponentCallbacks2, android.graphics.SurfaceTexture.OnFrameAvailableListener, android.view.KeyEvent.Callback, android.view.LayoutInflater.Factory, android.view.LayoutInflater.Factory2, android.view.View.OnCreateContextMenuListener, android.view.Window.Callback, EncodingConfig.EncodingCallback

public class BroadcastActivity
extends android.app.Activity
implements android.graphics.SurfaceTexture.OnFrameAvailableListener, EncodingConfig.EncodingCallback

Shows the camera preview on screen while simultaneously recording it to a .mp4 file.

Every time we receive a frame from the camera, we need to:

• Render the frame to the SurfaceView, on GLSurfaceView's renderer thread.
• Render the frame to the mediacodec's input surface, on the encoder thread, if recording is enabled.

At any given time there are four things in motion:

1. The UI thread, embodied by this Activity. We must respect -- or work around -- the app lifecycle changes. In particular, we need to release and reacquire the Camera so that, if the user switches away from us, we're not preventing another app from using the camera.
2. The Camera, which will busily generate preview frames once we hand it a SurfaceTexture. We'll get notifications on the main UI thread unless we define a Looper on the thread where the SurfaceTexture is created (the GLSurfaceView renderer thread).
3. The video encoder thread, embodied by TextureMovieEncoder. This needs to share the Camera preview external texture with the GLSurfaceView renderer, which means the EGLContext in this thread must be created with a reference to the renderer thread's context in hand.
4. The GLSurfaceView renderer thread, embodied by CameraSurfaceRenderer. The thread is created for us by GLSurfaceView. We don't get callbacks for pause/resume or thread startup/shutdown, though we could generate messages from the Activity for most of these things. The EGLContext created on this thread must be shared with the video encoder, and must be used to create a SurfaceTexture that is used by the Camera. As the creator of the SurfaceTexture, it must also be the one to call updateTexImage(). The renderer thread is thus at the center of a multi-thread nexus, which is a bit awkward since it's the thread we have the least control over.

GLSurfaceView is fairly painful here. Ideally we'd create the video encoder, create an EGLContext for it, and pass that into GLSurfaceView to share. The API doesn't allow this, so we have to do it the other way around. When GLSurfaceView gets torn down (say, because we rotated the device), the EGLContext gets tossed, which means that when it comes back we have to re-create the EGLContext used by the video encoder. (And, no, the "preserve EGLContext on pause" feature doesn't help.)

We could simplify this quite a bit by using TextureView instead of GLSurfaceView, but that comes with a performance hit. We could also have the renderer thread drive the video encoder directly, allowing them to work from a single EGLContext, but it's useful to decouple the operations, and it's generally unwise to perform disk I/O on the thread that renders your UI.

We want to access Camera from the UI thread (setup, teardown) and the renderer thread (configure SurfaceTexture, start preview), but the API says you can only access the object from a single thread. So we need to pick one thread to own it, and the other thread has to access it remotely. Some things are simpler if we let the renderer thread manage it, but we'd really like to be sure that Camera is released before we leave onPause(), which means we need to make a synchronous call from the UI thread into the renderer thread, which we don't really have full control over. It's less scary to have the UI thread own Camera and have the renderer call back into the UI thread through the standard Handler mechanism.

(The camera docs recommend accessing the camera from a non-UI thread to avoid bogging the UI thread down. Since the GLSurfaceView-managed renderer thread isn't a great choice, we might want to create a dedicated camera thread. Not doing that here.)

With three threads working simultaneously (plus Camera causing periodic events as frames arrive) we have to be very careful when communicating state changes. In general we want to send a message to the thread, rather than directly accessing state in the object.

To exercise the API a bit, the video encoder is required to survive Activity restarts. In the current implementation it stops recording but doesn't stop time from advancing, so you'll see a pause in the video. (We could adjust the timer to make it seamless, or output a "paused" message and hold on that in the recording, or leave the Camera running so it continues to generate preview frames while the Activity is paused.) The video encoder object is managed as a static property of the Activity.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BroadcastActivity.CameraHandler Handles camera operation requests from other threads.

Field Summary

Fields inherited from class android.app.Activity

DEFAULT_KEYS_DIALER, DEFAULT_KEYS_DISABLE, DEFAULT_KEYS_SEARCH_GLOBAL, DEFAULT_KEYS_SEARCH_LOCAL, DEFAULT_KEYS_SHORTCUT, FOCUSED_STATE_SET, RESULT_CANCELED, RESULT_FIRST_USER, RESULT_OK

Fields inherited from class android.content.Context

ACCESSIBILITY_SERVICE, ACCOUNT_SERVICE, ACTIVITY_SERVICE, ALARM_SERVICE, APP_OPS_SERVICE, AUDIO_SERVICE, BIND_ABOVE_CLIENT, BIND_ADJUST_WITH_ACTIVITY, BIND_ALLOW_OOM_MANAGEMENT, BIND_AUTO_CREATE, BIND_DEBUG_UNBIND, BIND_IMPORTANT, BIND_NOT_FOREGROUND, BIND_WAIVE_PRIORITY, BLUETOOTH_SERVICE, CAPTIONING_SERVICE, CLIPBOARD_SERVICE, CONNECTIVITY_SERVICE, CONSUMER_IR_SERVICE, CONTEXT_IGNORE_SECURITY, CONTEXT_INCLUDE_CODE, CONTEXT_RESTRICTED, DEVICE_POLICY_SERVICE, DISPLAY_SERVICE, DOWNLOAD_SERVICE, DROPBOX_SERVICE, INPUT_METHOD_SERVICE, INPUT_SERVICE, KEYGUARD_SERVICE, LAYOUT_INFLATER_SERVICE, LOCATION_SERVICE, MEDIA_ROUTER_SERVICE, MODE_APPEND, MODE_ENABLE_WRITE_AHEAD_LOGGING, MODE_MULTI_PROCESS, MODE_PRIVATE, MODE_WORLD_READABLE, MODE_WORLD_WRITEABLE, NFC_SERVICE, NOTIFICATION_SERVICE, NSD_SERVICE, POWER_SERVICE, PRINT_SERVICE, SEARCH_SERVICE, SENSOR_SERVICE, STORAGE_SERVICE, TELEPHONY_SERVICE, TEXT_SERVICES_MANAGER_SERVICE, UI_MODE_SERVICE, USB_SERVICE, USER_SERVICE, VIBRATOR_SERVICE, WALLPAPER_SERVICE, WIFI_P2P_SERVICE, WIFI_SERVICE, WINDOW_SERVICE

Fields inherited from interface android.content.ComponentCallbacks2

TRIM_MEMORY_BACKGROUND, TRIM_MEMORY_COMPLETE, TRIM_MEMORY_MODERATE, TRIM_MEMORY_RUNNING_CRITICAL, TRIM_MEMORY_RUNNING_LOW, TRIM_MEMORY_RUNNING_MODERATE, TRIM_MEMORY_UI_HIDDEN

Constructor Summary

Constructors

Constructor and Description
BroadcastActivity()

Method Summary

Methods

Modifier and Type	Method and Description
protected BroadcastActivity.CameraHandler	getCameraHandler()
protected TextureMovieEncoder	getsVideoEncoder()
protected void	handleFailedFrame(java.lang.String errorString)
protected void	handleSavedFrame(ScreenShot screenShot) When the frame has been saved the message object will contain the file path of the bitmap
protected void	handleSavingFrame(java.lang.String savingString)
void	muxerStatusUpdate(EncodingConfig.MUXER_STATE muxerState)
protected void	onCreate(android.os.Bundle savedInstanceState)
protected void	onDestroy()
void	onFrameAvailable(android.graphics.SurfaceTexture st)
protected void	onPause()
protected void	onResume()
protected void	saveFrame(ScreenShot screenShot) Sends a message to the encoder.
void	toggleRecordingHandler() onClick handler for "record" button.

Methods inherited from class android.app.Activity

addContentView, closeContextMenu, closeOptionsMenu, createPendingResult, dismissDialog, dispatchGenericMotionEvent, dispatchKeyEvent, dispatchKeyShortcutEvent, dispatchPopulateAccessibilityEvent, dispatchTouchEvent, dispatchTrackballEvent, dump, findViewById, finish, finishActivity, finishActivityFromChild, finishAffinity, finishFromChild, getActionBar, getApplication, getCallingActivity, getCallingPackage, getChangingConfigurations, getComponentName, getCurrentFocus, getFragmentManager, getIntent, getLastNonConfigurationInstance, getLayoutInflater, getLoaderManager, getLocalClassName, getMenuInflater, getParent, getParentActivityIntent, getPreferences, getRequestedOrientation, getSystemService, getTaskId, getTitle, getTitleColor, getVolumeControlStream, getWindow, getWindowManager, hasWindowFocus, invalidateOptionsMenu, isChangingConfigurations, isChild, isDestroyed, isFinishing, isImmersive, isTaskRoot, managedQuery, moveTaskToBack, navigateUpTo, navigateUpToFromChild, onActionModeFinished, onActionModeStarted, onActivityResult, onApplyThemeResource, onAttachedToWindow, onAttachFragment, onBackPressed, onChildTitleChanged, onConfigurationChanged, onContentChanged, onContextItemSelected, onContextMenuClosed, onCreateContextMenu, onCreateDescription, onCreateDialog, onCreateDialog, onCreateNavigateUpTaskStack, onCreateOptionsMenu, onCreatePanelMenu, onCreatePanelView, onCreateThumbnail, onCreateView, onCreateView, onDetachedFromWindow, onGenericMotionEvent, onKeyDown, onKeyLongPress, onKeyMultiple, onKeyShortcut, onKeyUp, onLowMemory, onMenuItemSelected, onMenuOpened, onNavigateUp, onNavigateUpFromChild, onNewIntent, onOptionsItemSelected, onOptionsMenuClosed, onPanelClosed, onPostCreate, onPostResume, onPrepareDialog, onPrepareDialog, onPrepareNavigateUpTaskStack, onPrepareOptionsMenu, onPreparePanel, onProvideAssistData, onRestart, onRestoreInstanceState, onRetainNonConfigurationInstance, onSaveInstanceState, onSearchRequested, onStart, onStop, onTitleChanged, onTouchEvent, onTrackballEvent, onTrimMemory, onUserInteraction, onUserLeaveHint, onWindowAttributesChanged, onWindowFocusChanged, onWindowStartingActionMode, openContextMenu, openOptionsMenu, overridePendingTransition, recreate, registerForContextMenu, removeDialog, reportFullyDrawn, requestWindowFeature, runOnUiThread, setContentView, setContentView, setContentView, setDefaultKeyMode, setFeatureDrawable, setFeatureDrawableAlpha, setFeatureDrawableResource, setFeatureDrawableUri, setFinishOnTouchOutside, setImmersive, setIntent, setProgress, setProgressBarIndeterminate, setProgressBarIndeterminateVisibility, setProgressBarVisibility, setRequestedOrientation, setResult, setResult, setSecondaryProgress, setTitle, setTitle, setTitleColor, setVisible, setVolumeControlStream, shouldUpRecreateTask, showDialog, showDialog, startActionMode, startActivities, startActivities, startActivity, startActivity, startActivityForResult, startActivityForResult, startActivityFromChild, startActivityFromChild, startActivityFromFragment, startActivityFromFragment, startActivityIfNeeded, startActivityIfNeeded, startIntentSender, startIntentSender, startIntentSenderForResult, startIntentSenderForResult, startIntentSenderFromChild, startIntentSenderFromChild, startManagingCursor, startNextMatchingActivity, startNextMatchingActivity, startSearch, stopManagingCursor, takeKeyEvents, triggerSearch, unregisterForContextMenu

Methods inherited from class android.view.ContextThemeWrapper

applyOverrideConfiguration, attachBaseContext, getResources, getTheme, setTheme

Methods inherited from class android.content.ContextWrapper

bindService, checkCallingOrSelfPermission, checkCallingOrSelfUriPermission, checkCallingPermission, checkCallingUriPermission, checkPermission, checkUriPermission, checkUriPermission, clearWallpaper, createConfigurationContext, createDisplayContext, createPackageContext, databaseList, deleteDatabase, deleteFile, enforceCallingOrSelfPermission, enforceCallingOrSelfUriPermission, enforceCallingPermission, enforceCallingUriPermission, enforcePermission, enforceUriPermission, enforceUriPermission, fileList, getApplicationContext, getApplicationInfo, getAssets, getBaseContext, getCacheDir, getClassLoader, getContentResolver, getDatabasePath, getDir, getExternalCacheDir, getExternalCacheDirs, getExternalFilesDir, getExternalFilesDirs, getFilesDir, getFileStreamPath, getMainLooper, getObbDir, getObbDirs, getPackageCodePath, getPackageManager, getPackageName, getPackageResourcePath, getSharedPreferences, getWallpaper, getWallpaperDesiredMinimumHeight, getWallpaperDesiredMinimumWidth, grantUriPermission, isRestricted, openFileInput, openFileOutput, openOrCreateDatabase, openOrCreateDatabase, peekWallpaper, registerReceiver, registerReceiver, removeStickyBroadcast, removeStickyBroadcastAsUser, revokeUriPermission, sendBroadcast, sendBroadcast, sendBroadcastAsUser, sendBroadcastAsUser, sendOrderedBroadcast, sendOrderedBroadcast, sendOrderedBroadcastAsUser, sendStickyBroadcast, sendStickyBroadcastAsUser, sendStickyOrderedBroadcast, sendStickyOrderedBroadcastAsUser, setWallpaper, setWallpaper, startInstrumentation, startService, stopService, unbindService, unregisterReceiver

Methods inherited from class android.content.Context

getString, getString, getText, obtainStyledAttributes, obtainStyledAttributes, obtainStyledAttributes, obtainStyledAttributes, registerComponentCallbacks, unregisterComponentCallbacks

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BroadcastActivity

public BroadcastActivity()

Method Detail

onCreate

protected void onCreate(android.os.Bundle savedInstanceState)

Overrides:

onCreate in class android.app.Activity

getsVideoEncoder

protected TextureMovieEncoder getsVideoEncoder()

onResume

protected void onResume()

Overrides:

onResume in class android.app.Activity

onPause

protected void onPause()

Overrides:

onPause in class android.app.Activity

onDestroy

protected void onDestroy()

Overrides:

onDestroy in class android.app.Activity

toggleRecordingHandler

public void toggleRecordingHandler()

onClick handler for "record" button.

onFrameAvailable

public void onFrameAvailable(android.graphics.SurfaceTexture st)

Specified by:

onFrameAvailable in interface android.graphics.SurfaceTexture.OnFrameAvailableListener

muxerStatusUpdate

public void muxerStatusUpdate(EncodingConfig.MUXER_STATE muxerState)

Specified by:

muxerStatusUpdate in interface EncodingConfig.EncodingCallback

saveFrame

protected void saveFrame(ScreenShot screenShot)

Sends a message to the encoder. The encoder is directly connected to the EGLSurface, it accepts messages and it guarantees the EGLContext's state so it's convenient to run the command via the encoder. All the user has to do is define a screenshot and call this method. The app takes care of the rest. Only customization he may want is to handle the frame messages (see below) to react to the status of the frame save.

handleFailedFrame

protected void handleFailedFrame(java.lang.String errorString)

handleSavedFrame

protected void handleSavedFrame(ScreenShot screenShot)

When the frame has been saved the message object will contain the file path of the bitmap

Parameters:

screenShot -

handleSavingFrame

protected void handleSavingFrame(java.lang.String savingString)

getCameraHandler

protected BroadcastActivity.CameraHandler getCameraHandler()