The Evolution of Android Camera APIs
8 min readJul 14, 2022
The Android camera has evolved over the years to offer multiple cogs and levers to developers to customize the camera viewing and picture-taking experience for their applications.
In modern applications, the camera has become an integral part of interfacing. Whether it is used for video calling, creating content, scanning QR/Bar codes, or playing AR games, having a good camera interface defines the success of the application in many cases. Content creation with filters and various other formats (reels, boomerang) adds quite some value to the application.
Any mobile developer who wants to implement a camera interface in their application needs to understand the intricacies and the techniques used for implementing it. Having good knowledge about the options and the available interfaces within the Android system empowers them to customize this interface and bring the best output for the application.
Support Matrix
Until now the Android SDK has three types of camera APIs.
- Camera1
- Camera2
- CameraX
Camera1 API supports API Level 9–20, Camera2 supports API Level 21 and above, and CameraX supports API Level 21 and above with backward compatibility built-in.
Camera1
The first version of the Android camera API has supported and served its role for all the camera-related functionalities that the hardware had supported for quite some time. Programmatically, the only thing to check is whether the device has a camera feature or not, and use permission in the manifest file.
To either preview or do the camera operations, use SurfaceView, TextureView to render content and instantiate an object of this class and implement the SurfaceTextureListener interface.
Then override the below methods to complete implementation.
/**
Invoked when a TextureView's SurfaceTexture is ready for use
**/
public void onSurfaceTextureAvailable(SurfaceTexture arg0, int arg1, int arg2) {}
/**
* Iinvoked when the specified SurfaceTexture is about to be destroyed.
*/
public boolean onSurfaceTextureDestroyed(SurfaceTexture arg0) {}
/**
* Invoked when the SurfaceTexture's buffers size changed
*/
public void onSurfaceTextureSizeChanged(SurfaceTexture arg0, int arg1,int arg2) {}
/*
* Invoked when the specified SurfaceTexture is updated through updateTexImage().
*/
public void onSurfaceTextureUpdated(SurfaceTexture arg0) {}As it is deprecated and not in usage, I am not discussing much here in this blog.
Camera2
With evolving camera technology, the Android framework has grown in capability with a wide range of customization options. For example, mobile manufacturers want to assemble multiple cameras for better user experiences and to achieve wide-angle previews, photography features, and multi-camera access at a time.
The Camera2 API is more complex and asynchronous, and has lots of capture controls, states, and metadata. But Camera2 API is more customizable and provides in-depth controls for complex uses like manual exposure (ISO, shutter speed), focus, RAW capture, etc.
Following are steps for using camera2 API
- Start from CameraManager
- Setup the output targets
- Get a CameraDevice
- Create a CaptureRequest from the CameraDevice
- Create a CaptureRequestSession from the CameraDevice
- Submit a CaptureRequest to CaptureRequestSession
- Get the CaptureResults
- Thread maintenance
There are multiple Camera2 API implementations depending on the needs available in so many sources.
Here is an example that captures without preview but runs in the background.
Background Service
/**
* JobIntentService that captures the picture without preview and runs in the background
*/
class NoPreviewCaptureService : JobIntentService() {
private val CAMERA = CameraCharacteristics.LENS_FACING_BACK // Can be changed. FACING_FRONT is selfie camera
private var mImageReader: ImageReader? = null
override fun onCreate() {
super.onCreate()
Log.i(TAG, "onCreate()")
startBackgroundThread()
}
/**
* Starts a background thread and its [Handler].
*/
private fun startBackgroundThread() {
mBackgroundThread = HandlerThread(TAG)
mBackgroundThread?.start()
mBackgroundHandler = Handler(mBackgroundThread!!.looper)
}
private val cameraStateCallback: CameraDevice.StateCallback =
object : CameraDevice.StateCallback() {
override fun onOpened(camera: CameraDevice) {
Log.i(TAG, "onOpened()")
mCameraDevice = camera
try {
camera.createCaptureSession(
listOf(mImageReader!!.surface),
sessionStateCallback,
null
)
} catch (e: Exception) {
Log.e(TAG, "createCaptureSession Failed" + Log.getStackTraceString(e))
}
}
override fun onDisconnected(camera: CameraDevice) {}
override fun onError(camera: CameraDevice, error: Int) {}
}
private val sessionStateCallback: CameraCaptureSession.StateCallback =
object : CameraCaptureSession.StateCallback() {
override fun onConfigured(session: CameraCaptureSession) {
mCaptureSession = session
capturePicture()
}
override fun onConfigureFailed(session: CameraCaptureSession) {}
}
/**
* This a callback object for the [ImageReader]. "onImageAvailable" will be called when a
* still image is ready to be saved.
*/
private val onImageAvailableListener = OnImageAvailableListener { reader ->
Log.i(TAG, "OnImageAvailableListener")
}
/**
* Return the Camera Id which matches the field CAMERA.
*/
private fun getCamera(manager: CameraManager): String? {
try {
for (cameraId in manager.cameraIdList) {
val characteristics = manager.getCameraCharacteristics(cameraId!!)
val cOrientation = characteristics.get(CameraCharacteristics.LENS_FACING)!!
if (cOrientation == CAMERA) {
return cameraId
}
}
} catch (e: CameraAccessException) {
e.printStackTrace()
}
return null
}
override fun onHandleWork(intent: Intent) {
Log.i(TAG, "onHandleWork()")
Handler(Looper.getMainLooper()).post {
val manager = getSystemService(Context.CAMERA_SERVICE) as CameraManager
try {
if (ActivityCompat.checkSelfPermission(
this, Manifest.permission.CAMERA
) != PackageManager.PERMISSION_GRANTED
) {
Log.i(TAG, "No Camera permissions.")
}
this?.let {
externalFilesFolder = it.getExternalFilesDir(null)?.absolutePath
}
manager.openCamera(getCamera(manager)!!, cameraStateCallback, null)
val characteristics = manager.getCameraCharacteristics(getCamera(manager)!!)
val map = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
// For still image captures, we use the largest available size.
val largest = Collections.max(
Arrays.asList(*map?.getOutputSizes(ImageFormat.JPEG)),
CompareSizesByArea()
)
mImageReader =
ImageReader.newInstance(largest.width, largest.height, ImageFormat.JPEG, 1)
mImageReader?.setOnImageAvailableListener(
onImageAvailableListener,
mBackgroundHandler
)
} catch (e: CameraAccessException) {
Log.e(TAG, e.localizedMessage)
}
}
}
/**
* Compares two `Size`s based on their areas.
*/
internal class CompareSizesByArea : Comparator {
// We cast here to ensure the multiplications won't overflow
override fun compare(lhs: Size, rhs: Size) =
signum(lhs.width.toLong() * lhs.height - rhs.width.toLong() * rhs.height)
}
/**
* Fragment methods
*/
companion object {
private val TAG = "CaptureService"
/**
* Conversion from screen rotation to JPEG orientation.
*/
private val ORIENTATIONS = SparseIntArray()
init {
ORIENTATIONS.append(Surface.ROTATION_0, 90)
ORIENTATIONS.append(Surface.ROTATION_90, 0)
ORIENTATIONS.append(Surface.ROTATION_180, 270)
ORIENTATIONS.append(Surface.ROTATION_270, 180)
}
// Reference to the path of external files folder
var externalFilesFolder: String? = null
val FILENAME_FORMAT = "yyyy-MM-dd-HH-mm-ss-SSS"
fun enqueueWork(context: Context, intent: Intent) {
enqueueWork(context, NoPreviewCaptureService::class.java, 1, intent)
}
private fun releaseResources() {
Log.i(TAG, "Releasing Resources")
closeCamera()
stopBackgroundThread()
}
private var mCaptureSession: CameraCaptureSession? = null
private var mBackgroundThread: HandlerThread? = null
private var mBackgroundHandler: Handler? = null
private var mCameraDevice: CameraDevice? = null
/**
* Stops the background thread and its [Handler].
*/
private fun stopBackgroundThread() {
mBackgroundThread?.quitSafely()
mBackgroundThread = null
mBackgroundHandler?.removeCallbacksAndMessages(null)
mBackgroundHandler = null
}
/**
* Closes the current [CameraDevice].
*/
private fun closeCamera() {
// stop any existing captures
try {
mCaptureSession?.abortCaptures()
mCaptureSession?.stopRepeating()
} catch (e: CameraAccessException) {
Log.i(TAG, "Camera session cannot abort")
}
// Close the camera device
mCameraDevice?.close()
mCaptureSession?.close()
mCameraDevice = null
mCaptureSession = null
// Close the capture session
}
}
/**
* Capture request starts here
*/
fun capturePicture() {
if (mCameraDevice == null) {
// Already closed
Log.i(TAG, "Camera is not available.")
return
}
try {
val captureRequest =
mCameraDevice?.createCaptureRequest(CameraDevice.TEMPLATE_STILL_CAPTURE)
captureRequest?.let {
it.set(CaptureRequest.CONTROL_MODE, CaptureRequest.CONTROL_MODE_OFF)
it.set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_OFF)
it.set(CaptureRequest.SENSOR_SENSITIVITY, 100)
val exposureTime = 1000 * 1000 * 1000 / 20.toLong()
it.set(CaptureRequest.SENSOR_EXPOSURE_TIME, exposureTime)
it.set(CaptureRequest.JPEG_ORIENTATION, 90)
mImageReader?.let { imageReader ->
Log.i(TAG, "Assigning capture stuff")
it.addTarget(imageReader.surface)
// Need handler here for
imageReader.setOnImageAvailableListener(
mOnImageAvailableListener,
mBackgroundHandler
)
}
mCaptureSession?.capture(
captureRequest.build(),
object : CameraCaptureSession.CaptureCallback() {
override fun onCaptureCompleted(
session: CameraCaptureSession,
request: CaptureRequest,
result: TotalCaptureResult
) {
super.onCaptureCompleted(session, request, result)
// We are done
Log.i(TAG, "Completed taking all pictures. Move ahead.")
}
},
mBackgroundHandler
)
}
} catch (e: Exception) {
e.printStackTrace()
Log.e(TAG, "Capture Session Exception: " + Log.getStackTraceString(e))
}
}
/**
* This a callback object for the [ImageReader]. "onImageAvailable" will be called when a
* still image is ready to be saved.
*/
private val mOnImageAvailableListener = OnImageAvailableListener { reader ->
Log.i(TAG, "Image is available")
mBackgroundHandler?.post(ImageHandler(reader?.acquireNextImage()))
}
/**
* Saves a JPEG [Image] into the specified [File].
*/
class ImageHandler(image: Image?) : Runnable {
private val mImage = image
override fun run() {
mImage?.let { theImage ->
// Get the byte buffer
val buffer = theImage.planes[0].buffer
val bytes = ByteArray(buffer.remaining()).apply { buffer.get(this) }
// Create time-stamped output file to hold the image
val photoFile = File(
externalFilesFolder,
SimpleDateFormat(
FILENAME_FORMAT, Locale.US
).format(System.currentTimeMillis()) + ".jpg"
)
var output: FileOutputStream? = null
try {
output = FileOutputStream(photoFile).apply {
write(bytes)
}
val msg = "Photo saved : ${photoFile.absolutePath}"
Log.d(TAG, msg)
} catch (e: IOException) {
Log.e(TAG, e.toString())
} finally {
mImage?.close()
output?.let {
try {
it.close()
} catch (e: IOException) {
Log.e(TAG, e.toString())
}
releaseResources()
}
}
}
}
}
}
}Calling Service
val intent = Intent(this, NoPreviewCaptureService::class.java)
NoPreviewCaptureService.enqueueWork(this, intent)Declare this service in the manifest.xml
<service android:name=".NoPreviewCaptureService"
android:exported="true"
android:permission="android.permission.BIND_JOB_SERVICE" />CameraX
CameraX is the latest solution for developers who are creating highly specialized functionality for low-level control of the capture flow and when more customization is required. CameraX is built on top of the Camera2 API only.
Advantages:
- Easy to use compared to Camera2
- Backward compatibility as it is part of Android Jetpack to Android 5.0. and is consistent across devices
- Lifecycle aware, no need to maintain open, close the camera or when to create capture sessions
Below is a simple example that captures the picture and camera switch with CameraX.
Declare below dependencies in build.gradle
def camerax_version = "1.0.0-beta12"
// CameraX core library using camera2 implementation
implementation "androidx.camera:camera-camera2:$camerax_version"
// CameraX Lifecycle Library
implementation "androidx.camera:camera-lifecycle:$camerax_version"
// CameraX View class
implementation "androidx.camera:camera-view:1.0.0-alpha19"Activity Code
class MainActivity : AppCompatActivity() {
private var imageCapture: ImageCapture? = null
private lateinit var outputDirectory: File
private lateinit var cameraExecutor: ExecutorService
private val REQUIRED_PERMISSIONS = arrayOf(Manifest.permission.CAMERA)
private val REQUEST_CODE_PERMISSIONS = 20
// Setting Back camera as a default
private var cameraSelector = CameraSelector.DEFAULT_BACK_CAMERA
private val FILENAME_FORMAT = "yyyy-MM-dd-HH-mm-ss-SSS"
val TAG = "CameraXCapture"
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
requestPermission()
// Set up the listener for take photo button
camera_capture_button.setOnClickListener { takePhoto() }
switch_btn.setOnClickListener {
//change the cameraSelector
cameraSelector = if (cameraSelector == CameraSelector.DEFAULT_BACK_CAMERA) {
CameraSelector.DEFAULT_FRONT_CAMERA
} else {
CameraSelector.DEFAULT_BACK_CAMERA
}
// restart the camera
startCamera()
}
outputDirectory = getOutputDirectory()
cameraExecutor = Executors.newSingleThreadExecutor()
}
private fun takePhoto() {
// Get a stable reference of the modifiable image capture use case
val imageCapture = imageCapture ?: return
// Create time-stamped output file to hold the image
val photoFile = File(
outputDirectory,
SimpleDateFormat(
FILENAME_FORMAT, Locale.US
).format(System.currentTimeMillis()) + ".jpg"
)
// Create output options object which contains file + metadata
val outputOptions = ImageCapture.OutputFileOptions.Builder(photoFile).build()
// Set up image capture listener, which is triggered after photo has
// been taken
imageCapture.takePicture(
outputOptions,
ContextCompat.getMainExecutor(this),
object : ImageCapture.OnImageSavedCallback {
override fun onError(exc: ImageCaptureException) {
Log.e(TAG, "Photo capture failed: ${exc.message}", exc)
}
override fun onImageSaved(output: ImageCapture.OutputFileResults) {
val savedUri = Uri.fromFile(photoFile)
val msg = "Photo capture succeeded: $savedUri"
Toast.makeText(baseContext, msg, Toast.LENGTH_SHORT).show()
Log.d(TAG, msg)
}
})
}
private fun startCamera() {
val cameraProviderFuture = ProcessCameraProvider.getInstance(this)
cameraProviderFuture.addListener({
// Used to bind the lifecycle of cameras to the lifecycle owner
val cameraProvider: ProcessCameraProvider = cameraProviderFuture.get()
// Preview
val preview = Preview.Builder().build().also {
it.setSurfaceProvider(viewFinder.surfaceProvider)
}
imageCapture = ImageCapture.Builder().build()
try {
// Unbind use cases before rebinding
cameraProvider.unbindAll()
// Bind use cases to camera
cameraProvider.bindToLifecycle(
this, cameraSelector, preview, imageCapture
)
} catch (exc: Exception) {
Log.e(TAG, "Use case binding failed", exc)
}
}, ContextCompat.getMainExecutor(this))
}
private fun getOutputDirectory(): File {
val mediaDir = externalMediaDirs.firstOrNull()?.let {
File(it, resources.getString(R.string.app_name)).apply { mkdirs() }
}
return if (mediaDir != null ∧∧ mediaDir.exists())
mediaDir else filesDir
}
override fun onDestroy() {
super.onDestroy()
cameraExecutor.shutdown()
}
private fun requestPermission() {
if (allPermissionsGranted()) {
startCamera()
} else {
ActivityCompat.requestPermissions(this, REQUIRED_PERMISSIONS, REQUEST_CODE_PERMISSIONS)
}
}
private fun allPermissionsGranted() = REQUIRED_PERMISSIONS.all {
ContextCompat.checkSelfPermission(baseContext, it) == PackageManager.PERMISSION_GRANTED
}
override fun onRequestPermissionsResult(
requestCode: Int,
permissions: Array out String,
grantResults: IntArray
) {
super.onRequestPermissionsResult(requestCode, permissions, grantResults)
if (requestCode == REQUEST_CODE_PERMISSIONS) {
// If all permissions granted , then start Camera
if (allPermissionsGranted()) {
startCamera()
} else {
// If permissions are not granted,
// present a toast to notify the user that
// the permissions were not granted.
Toast.makeText(this, "Permissions not granted by the user.", Toast.LENGTH_SHORT)
.show()
finish()
}
}
}
}}XML Code
<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity">
<Button
android:id="@+id/camera_capture_button"
android:layout_width="100dp"
android:layout_height="100dp"
android:layout_marginBottom="50dp"
android:elevation="2dp"
android:scaleType="fitCenter"
android:text="@string/take_photo"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintLeft_toLeftOf="parent"
app:layout_constraintRight_toRightOf="parent" />
<Button
android:id="@+id/switch_btn"
android:layout_width="100dp"
android:layout_height="100dp"
android:layout_marginBottom="50dp"
android:elevation="2dp"
android:scaleType="fitCenter"
android:text="Switch"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintEnd_toStartOf="@id/camera_capture_button"
app:layout_constraintStart_toStartOf="parent" />
<androidx.camera.view.PreviewView
android:id="@+id/viewFinder"
android:layout_width="match_parent"
android:layout_height="match_parent" />
</androidx.constraintlayout.widget.ConstraintLayout>Mobile cameras will continue to evolve. Wide cameras, ultra-wide cameras, macro cameras, dual, triple, quad, and Penta back cameras, among other types of cameras, are now often found in mobile phones.
Note: — The blog was originally published on Innominds blogs
