I am trying to record a video from the camera, through AVCapture and AVAssetWriter, and to overlay some text and an image on the final video but I cannot achieve good performance and the device thermal throttles quite fast.
In order to do so I get the buffer from
func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
if output == videoDataOutput && self.videoInput.isReadyForMoreMediaData && self.isSessionStarted {
guard let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {
return
}
...
// i write the new buffer
var newimageBuffer = overlayFunction(image: self.mp, toBuffer: imageBuffer)
let newsb = try CMSampleBuffer(imageBuffer: newimageBuffer, formatDescription: sampleBuffer.formatDescription!, sampleTiming: newsampletiming)
// and i append it
self.videoInput.append(newsb)
...
}
}
And I draw on it through this function
func overlayFunction(image mp:UIImage?, toBuffer pixelBuffer: CVImageBuffer) -> CVImageBuffer{
guard
CVPixelBufferLockBaseAddress(pixelBuffer, CVPixelBufferLockFlags(rawValue: 0)) == kCVReturnSuccess,
let context = CGContext(data: CVPixelBufferGetBaseAddress(pixelBuffer),
width: CVPixelBufferGetWidth(pixelBuffer),
height: CVPixelBufferGetHeight(pixelBuffer),
bitsPerComponent: 8,
bytesPerRow: CVPixelBufferGetBytesPerRow(pixelBuffer),
space: CGColorSpaceCreateDeviceRGB(),
bitmapInfo: CGBitmapInfo(rawValue: CGBitmapInfo.byteOrder32Little.rawValue|CGImageAlphaInfo.premultipliedFirst.rawValue).rawValue)
else{
return pixelBuffer
}
UIGraphicsPushContext(context)
context.scaleBy(x: 1.0, y:-1.0)
context.setFillColor(CGColor(red: 255, green: 255, blue: 255, alpha: 255))
mp.draw(in: CGRect(...), blendMode: CGBlendMode.normal, alpha: 1)
context.scaleBy(x: 1.0, y: 1.0)
let attributes : [ NSAttributedString.Key : Any ] = [.font : UIFont(name: "Helvetica", size: 36.0), .foregroundColor : UIColor(red: 255, green: 255, blue: 255, alpha: 0.8), .strokeWidth : -5]
var stringWithAttributes = NSAttributedString(string: str, attributes: attributes)
stringWithAttributes.draw(at: CGPoint(...))
UIGraphicsPopContext()
CVPixelBufferUnlockBaseAddress(pixelBuffer, CVPixelBufferLockFlags(rawValue: 0))
return pixelBuffer
}
However the performance is terrible. The resulting video lags a lot. What could I do in order to improve its speed?
Thanks for any help!