OpenHarmony USB UVC camera 适配说明

 

摘要

本文通过OpenHarmony 3.2.2 Release RK3568适配logitech webcam C925e以及杰锐微通 usb camera适配实践操作，梳理了OpenHarmony适配UVC协议的USB camera的方法。

本文描述的OpenHarmony UVC协议usb camera适配说明，不止适用于RK3568以及logitech webcam C925e usb camera以及杰锐微通 usb camera，除了图像硬编码与芯片硬编码库相关外，其它均使用的通用UVC协议和OpenHarmony 通用框架。

 

一、OpenHarmony 3.2.2 Release camera框架以及对USB camera的支持介绍

OpenHarmony 3.2.2 Release camera_host代码里预留了UVC camera的适配框架。kernel/linux/linux-5.10/drivers/media/usb/uvc下提供了uvc协议驱动。OpenHarmony 3.2.2 Release版本当前camera框架支持配置一个摄像头实现预览、拍照、录像功能。两个以及多个摄像头的差异配置以及多摄的动态切换功能当前还在增补完善中。如果要实现多摄差异配置以及多摄动态切换功能，适配改动较多，需要自行适配增补。

注：当前适配修改说明：当前方案使用USB camera驱动替掉了非USB camera的配置，所以适配后，非USB camera无法使用，更全面的方案请关注社区camera框架进展。如果后续方案有进一步改进，会在这里更新。

 

二、OpenHarmony USB UVC camera的适配步骤说明以及适配修改：

 

1、确定Linux 内核UVC协议已生效以及USB UVC camera被识别：

# lsusb
Bus 005 Device 001: ID 1d6b:0002
Bus 003 Device 001: ID 1d6b:0001
Bus 001 Device 001: ID 1d6b:0002
Bus 006 Device 001: ID 1d6b:0003
Bus 001 Device 002: ID 046d:085b###插上logitech webcam C925e新增了如下USB ID
Bus 004 Device 001: ID 1d6b:0001
Bus 002 Device 001: ID 1d6b:0002
#
# cat /sys/kernel/debug/usb/devices
......
T:  Bus=01 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#=  2 Spd=480  MxCh= 0
D:  Ver= 2.00 Cls=ef(misc ) Sub=02 Prot=01 MxPS=64 #Cfgs=  1
P:  Vendor=046d ProdID=085b Rev= 0.16
S:  Product=Logitech Webcam C925e//识别到的设备型号
S:  SerialNumber=9D5B375F
C:* #Ifs= 4 Cfg#= 1 Atr=80 MxPwr=500mA
A:  FirstIf#= 0 IfCount= 2 Cls=0e(video) Sub=03 Prot=00
A:  FirstIf#= 2 IfCount= 2 Cls=01(audio) Sub=02 Prot=00
I:* If#= 0 Alt= 0 #EPs= 1 Cls=0e(video) Sub=01 Prot=00 Driver=uvcvideo
E:  Ad=83(I) Atr=03(Int.) MxPS=  64 Ivl=16ms
I:* If#= 1 Alt= 0 #EPs= 0 Cls=0e(video) Sub=02 Prot=00 Driver=uvcvideo
I:  If#= 1 Alt= 1 #EPs= 1 Cls=0e(video) Sub=02 Prot=00 Driver=uvcvideo
E:  Ad=81(I) Atr=05(Isoc) MxPS= 192 Ivl=125us
I:  If#= 1 Alt= 2 #EPs= 1 Cls=0e(video) Sub=02 Prot=00 Driver=uvcvideo
E:  Ad=81(I) Atr=05(Isoc) MxPS= 384 Ivl=125us
I:  If#= 1 Alt= 3 #EPs= 1 Cls=0e(video) Sub=02 Prot=00 Driver=uvcvideo
E:  Ad=81(I) Atr=05(Isoc) MxPS= 512 Ivl=125us
I:  If#= 1 Alt= 4 #EPs= 1 Cls=0e(video) Sub=02 Prot=00 Driver=uvcvideo
E:  Ad=81(I) Atr=05(Isoc) MxPS= 640 Ivl=125us
I:  If#= 1 Alt= 5 #EPs= 1 Cls=0e(video) Sub=02 Prot=00 Driver=uvcvideo
E:  Ad=81(I) Atr=05(Isoc) MxPS= 800 Ivl=125us
I:  If#= 1 Alt= 6 #EPs= 1 Cls=0e(video) Sub=02 Prot=00 Driver=uvcvideo
E:  Ad=81(I) Atr=05(Isoc) MxPS= 944 Ivl=125us
I:  If#= 1 Alt= 7 #EPs= 1 Cls=0e(video) Sub=02 Prot=00 Driver=uvcvideo
E:  Ad=81(I) Atr=05(Isoc) MxPS=1280 Ivl=125us
I:  If#= 1 Alt= 8 #EPs= 1 Cls=0e(video) Sub=02 Prot=00 Driver=uvcvideo
E:  Ad=81(I) Atr=05(Isoc) MxPS=1600 Ivl=125us
I:  If#= 1 Alt= 9 #EPs= 1 Cls=0e(video) Sub=02 Prot=00 Driver=uvcvideo
E:  Ad=81(I) Atr=05(Isoc) MxPS=1984 Ivl=125us
I:  If#= 1 Alt=10 #EPs= 1 Cls=0e(video) Sub=02 Prot=00 Driver=uvcvideo
E:  Ad=81(I) Atr=05(Isoc) MxPS=2688 Ivl=125us
I:  If#= 1 Alt=11 #EPs= 1 Cls=0e(video) Sub=02 Prot=00 Driver=uvcvideo
E:  Ad=81(I) Atr=05(Isoc) MxPS=3060 Ivl=125us
I:* If#= 2 Alt= 0 #EPs= 0 Cls=01(audio) Sub=01 Prot=00 Driver=(none)
I:* If#= 3 Alt= 0 #EPs= 0 Cls=01(audio) Sub=02 Prot=00 Driver=(none)
I:  If#= 3 Alt= 1 #EPs= 1 Cls=01(audio) Sub=02 Prot=00 Driver=(none)
E:  Ad=82(I) Atr=05(Isoc) MxPS=  68 Ivl=1ms
I:  If#= 3 Alt= 2 #EPs= 1 Cls=01(audio) Sub=02 Prot=00 Driver=(none)
E:  Ad=82(I) Atr=05(Isoc) MxPS= 100 Ivl=1ms
I:  If#= 3 Alt= 3 #EPs= 1 Cls=01(audio) Sub=02 Prot=00 Driver=(none)
E:  Ad=82(I) Atr=05(Isoc) MxPS= 132 Ivl=1ms
......
#
# ls -l /dev/video*
crwxrwxrwx 1 camera_host camera_host 81,   0 2017-08-10 20:30 /dev/video0
crwxrwxrwx 1 camera_host camera_host 81,   1 2017-08-10 20:30 /dev/video1
crwxrwxrwx 1 root        root        81,  14 2017-08-10 20:30 /dev/video10//插上logitech webcam C925e新增了video9、video10和media1
crwxrwxrwx 1 camera_host camera_host 81,   2 2017-08-10 20:30 /dev/video2
crwxrwxrwx 1 camera_host camera_host 81,   3 2017-08-10 20:30 /dev/video3
crwxrwxrwx 1 camera_host camera_host 81,   4 2017-08-10 20:30 /dev/video4
crwxrwxrwx 1 camera_host camera_host 81,   5 2017-08-10 20:30 /dev/video5
crwxrwxrwx 1 camera_host camera_host 81,   6 2017-08-10 20:30 /dev/video6
crwxrwxrwx 1 camera_host camera_host 81,   7 2017-08-10 20:30 /dev/video7
crwxrwxrwx 1 camera_host camera_host 81,   8 2017-08-10 20:30 /dev/video8
crwxrwxrwx 1 root        root        81,  13 2017-08-10 20:30 /dev/video9//插上logitech webcam C925e新增了video9、video10和media1
#
注意如果USB移除有问题，会发现一旦该两video节点新增以后，未移除，再次插上时ls -l /dev/video*设备节点无变化的情况。

2、确认UVC驱动名称：

OpenHarmony linux-5.10内核默认UVC驱动，驱动名称为uvcvideo。自行移植的Linux内核版本，查看驱动名称：

kernel/linux/linux-5.10/drivers/media/usb/uvc/uvc_driver.c
......
struct uvc_driver uvc_driver = {
    .driver = {
        .name       = "uvcvideo",//--------uvc驱动名称---------
        .probe      = uvc_probe,
        .disconnect = uvc_disconnect,
        .suspend    = uvc_suspend,
        .resume     = uvc_resume,
        .reset_resume   = uvc_reset_resume,
        .id_table   = uvc_ids,
        .supports_autosuspend = 1,
    },
};
​
static int __init uvc_init(void)
{
    int ret;
​
    uvc_debugfs_init();
​
    ret = usb_register(&uvc_driver.driver);
    if (ret < 0) {
        uvc_debugfs_cleanup();
        return ret;
    }
​
    printk(KERN_INFO DRIVER_DESC " (" DRIVER_VERSION ")\n");
    return 0;
}
......

3、对usb camera动态建立的video设备节点增加读写权限：

1、base/startup/init/ueventd/etc/ueventd.config
/dev/rtc0 0640 3819 3819
/dev/tty0 0660 0 1000
/dev/uinput 0660 3011 3011
/dev/access_token_id 0666 3020 3020
/dev/block/sdd19 0660 6666 6666
/dev/watchdog 0660  watchdog watchdog
/dev/hdf_input_event* 0660 3029 3029
/dev/HDF* 0666 0 0
/dev/ttyS* 0666 0 0
/dev/ttyACM* 0666 0 0
/dev/ttyUSB* 0666 0 0
# 如下,增补usb camera动态建立的video设备节点的读写权限
/dev/video* 0777 0 0
​
2、base/security/selinux/sepolicy/base/te/ueventd.te
......
allow ueventd dev_v_file:chr_file { relabelto };
allow ueventd dev_vhci_file:chr_file { relabelto };
#allow ueventd dev_video_file:chr_file { relabelto };
这里如下：注释掉上面的relabelto权限行，新增unlink getattr setattr权限
allow ueventd dev_video_file:chr_file { unlink getattr setattr relabelto };
allow ueventd dev_vndbinder_file:chr_file { relabelto };
......
allow ueventd paramservice_socket:sock_file { write };
allow ueventd kernel:unix_stream_socket { connectto };
#这里如下:新增ueventd musl_param:file read权限
allow ueventd musl_param:file { read };
​
3、base/security/selinux/sepolicy/base/te/camera_host.te
......
allow camera_host dev_dma_heap_file:chr_file { read };
allow camera_host dev_dma_heap_file:chr_file { open };
allow camera_host dev_dma_heap_file:chr_file { ioctl };
allow camera_host dev_video_file:chr_file { map };
#这里如下:添加musl_param:file的read open权限
allow camera_host musl_param:file { read open };
#这里如下:添加data_file:dir的search read open write权限
allow camera_host data_file:dir { search read open write };
#这里如下:添加proc_version_file:file的getattr权限
allow camera_host proc_version_file:file { getattr };
allowxperm camera_host dev_hdf_kevent:chr_file ioctl { 0x6201 0x6202 0x6203 };
#这里如下:添加更多的ioctl权限
allowxperm camera_host dev_video_file:chr_file ioctl { 0x5600 0x5602 0x5624 0x5625 0x561b 0x564a };
​
4、base/security/selinux/sepolicy/base/te/udevd.te
......
allow udevd data_file:file { ioctl read };
allow udevd data_file:sock_file { create unlink };
allow udevd data_udev:dir { add_name create getattr open read remove_name search watch write };
#这里如下:添加data_udev:file 权限
#allow udevd data_udev:file { create ioctl open read rename unlink write write open };
allow udevd data_udev:file { create ioctl open read rename unlink write write open getattr };
allow udevd data_udev:sock_file { create unlink };
allow udevd debug_param:file { map open read };
allow udevd default_param:file { map open read };
......

 

 

4、确定摄像头支持的像素格式以及分辨率：

开启调试日志打印，插上USB camera后下载日志查看摄像头是否支持V4L2SearchFormat函数里的VIDIOC_ENUM_FMT和VIDIOC_ENUM_FRAMESIZES格式信息查询，如果支持格式信息查询，则可以查询到摄像头支持的图像格式以及分辨率。如果不支持，则日志中会有如下日志打印：V4L2SearchFormat() no valid supported formats

 

如果VIDIOC_ENUM_FMT和VIDIOC_ENUM_FRAMESIZES格式信息查询不支持，下载Windows下usb camera工具软件，将摄像头插到USB口，通过工具软件查看USB UVC camera设备支持的像素格式以及分辨率：如下，说明logitech webcam C925e仅支持YUY2图像格式，分辩率也支持很多，包括640*480

 

 

5、修改绑定camera id与驱动名称：

device/board/hihope/rk3568/camera/device_manager/include/project_hardware.h
......
std::vector<HardwareConfiguration> hardware = {
    {CAMERA_FIRST, DM_M_SENSOR, DM_C_SENSOR, (std::string) "uvcvideo"},//修改rkisp_v5为uvcvideo
    {CAMERA_FIRST, DM_M_ISP, DM_C_ISP, (std::string) "isp"},
    {CAMERA_FIRST, DM_M_FLASH, DM_C_FLASH, (std::string) "flash"},
    {CAMERA_SECOND, DM_M_SENSOR, DM_C_SENSOR, (std::string) "uvcvideo1"},//修改Imx600为uvcvideo1
    {CAMERA_SECOND, DM_M_ISP, DM_C_ISP, (std::string) "isp1"},//修改isp为isp1
    {CAMERA_SECOND, DM_M_FLASH, DM_C_FLASH, (std::string) "flash1"}//修改flash为flash1
};
......

6、在数据源启动时指定像素格式：

drivers/peripheral/camera/hal/adapter/platform/v4l2/src/pipeline_core/nodes/v4l2_source_node/v4l2_source_node.cpp
......
RetCode V4L2SourceNode::Start(const int32_t streamId)
{
    RetCode rc = RC_OK;
    deviceManager_ = IDeviceManager::GetInstance();
    if (deviceManager_ == nullptr) {
        CAMERA_LOGE("get device manager failed.");
        return RC_ERROR;
    }
    rc = GetDeviceController();
    if (rc == RC_ERROR) {
        CAMERA_LOGE("GetDeviceController failed.");
        return RC_ERROR;
    }
    std::vector<std::shared_ptr<IPort>> outPorts = GetOutPorts();
    for (const auto& it : outPorts) {
        DeviceFormat format;
        format.fmtdesc.pixelformat = V4L2_PIX_FMT_YUYV;
        //修改原来指定的数据源像素格式V4L2_PIX_FMT_YUV420,V4L2_PIX_FMT_NV21为V4L2_PIX_FMT_YUYV（YUY2）。
        format.fmtdesc.width = it->format_.w_;
        format.fmtdesc.height = it->format_.h_;
        int bufCnt = it->format_.bufferCount_;
        rc = sensorController_->Start(bufCnt, format);
        if (rc == RC_ERROR) {
            CAMERA_LOGE("start failed.");
            return RC_ERROR;
        }
    }
    rc = SourceNode::Start(streamId);
    return rc;
}
......

7、在图像数据编码时指定源像素格式：

device/board/hihope/rk3568/camera/pipeline_core/src/node/rk_codec_node.cpp
......
void RKCodecNode::Yuv420ToRGBA8888(std::shared_ptr<IBuffer>& buffer)
{
    ......
    RockchipRga rkRga;
    ......
​
    rga_set_rect(&src.rect, 0, 0, buffer->GetWidth(), buffer->GetHeight(),
        buffer->GetWidth(), buffer->GetHeight(), RK_FORMAT_YUYV_422);
    //---修改YUV数据编码为RGBA预览数据时的源数据像素格式为RK_FORMAT_YUYV_422----
    /*rga_set_rect(&src.rect, 0, 0, buffer->GetWidth(), buffer->GetHeight(),
        buffer->GetWidth(), buffer->GetHeight(), RK_FORMAT_YCbCr_420_P);*/
    rga_set_rect(&dst.rect, 0, 0, buffer->GetWidth(), buffer->GetHeight(),
        buffer->GetWidth(), buffer->GetHeight(), RK_FORMAT_RGBA_8888);
​
    rkRga.RkRgaBlit(&src, &dst, NULL);
    rkRga.RkRgaFlush();
    free(temp);
}
​
void RKCodecNode::Yuv420ToJpeg(std::shared_ptr<IBuffer>& buffer)
{
    ......
    rga_set_rect(&src.rect, 0, 0, previewWidth_, previewHeight_,
        previewWidth_, previewHeight_, RK_FORMAT_YUYV_422);
    //---修改YUV数据编码为Jpeg拍照数据时的源数据像素格式为RK_FORMAT_YUYV_422----
    /*rga_set_rect(&src.rect, 0, 0, buffer->GetWidth(), buffer->GetHeight(),
        buffer->GetWidth(), buffer->GetHeight(), RK_FORMAT_YCbCr_420_P);*/
    rga_set_rect(&dst.rect, 0, 0, previewWidth_, previewHeight_,
        previewWidth_, previewHeight_, RK_FORMAT_RGB_888);
​
    rkRga.RkRgaBlit(&src, &dst, NULL);
    rkRga.RkRgaFlush();
    encodeJpegToMemory((unsigned char *)temp, previewWidth_, previewHeight_, nullptr, &jpegSize, &jBuf);
    ......
}

//rk_codec_node.h添加USB Camera YUV422转YUV420（YYYYUVUV）的转换函数定义
device/board/hihope/rk3568/camera/pipeline_core/src/node/rk_codec_node.h
void YUYV422ToYuv420(u_char* dst, u_char* src, int width, int height);

//rk_codec_node.cpp添加USB Camera YUV422转YUV420（YYYYUVUV）的转换函数
void RKCodecNode::YUYV422ToYuv420(u_char* dst, u_char* src, int width, int height)
{
    int size = width * height;
    int pos = 0;
    int n = 0;
    for(int j = 0;j < height; j++) {
        pos = j * width * 2;
        for(int i = 0;i < width * 2; i += 2) {
            dst[n] = src[pos + i];
            n++;
        }
    }
    for(int j = 0;j < height; j += 2) {
        pos = j * width * 2;
        for(int i = 1;i < width * 2; i += 4) {
            dst[n] = src[pos + i];
            n++;
            dst[n] = src[pos + i + 2];
            n++;
        }
    }
}

void RKCodecNode::Yuv420ToH264(std::shared_ptr<IBuffer>& buffer)
{
    ......
    //下面这几行是在录像编码H264前，先将数据转换为YUV420，硬编码库能正常使用的格式。试了其它格式，发现编码出来的录像文件均不正常，采用了这种临时验证方法
    uint32_t width = buffer->GetWidth();
    uint32_t height = buffer->GetHeight();
    unsigned int size = width * height * 3 /2;
    u_char* buffer_cache = (u_char*)malloc(size);
    if (!buffer_cache) {
        CAMERA_LOGI("RKCodecNode::Yuv420ToH264 buffer_cache is nullptr\n");
        return;
    }
    YUYV422ToYuv420(buffer_cache, (u_char*)buffer->GetVirAddress(), width, height)；
    int nret = 0;
    nret = memcpy_s((void *)buffer->GetVirAddress(), buffer->GetSize(), (void *)buffer_cache, size);
    free(buffer_cache);
    if (nret != 0) {
        CAMERA_LOGI("RKCodecNode::Yuv420ToH264 memcpy_s != 0\n");
        return;
    }
    //上面这几行是在录像编码H264前，先将数据转换为YUV420，硬编码库能正常使用的格式。试了其它格式，发现编码出来的录像文件均不正常，采用了这种临时验证方法
    size_t buf_size = 0;
    struct timespec ts = {};
    ......
    if (mppStatus_ == 0) {
        MpiEncTestArgs args = {};
        args.width       = previewWidth_;
        args.height      = previewHeight_;
        args.format      = MPP_FMT_YUV420SP_VU;
        //---修改编码为H264录像数据时的源数据像素格式为MPP_FMT_YUV420SP_VU,试了其它枚举想直接硬编码，录像视频均不正常，故自己先行转为420,再硬编码
        //注：目前尝试了device/soc/rockchip/rk3568/hardware/mpp/include/mpp_frame.h下所有的YUV数据格式枚举，均未找到支持USB camera 输出的                             //YUV422_YUYV格式数据转H264编码后画面正常的枚举。尝试将数据自行转换为420格式后，指定枚举为MPP_FMT_YUV420SP_VU，录像画面正常。
        /*args.format      = MPP_FMT_YUV420P;*/
        args.type        = MPP_VIDEO_CodingAVC;
        halCtx_ = hal_mpp_ctx_create(&args);
        if (halCtx_ == nullptr) {
            CAMERA_LOGI("RKCodecNode::Yuv420ToH264 halCtx_ = %{public}p\n", halCtx_);
            return;
        }
        mppStatus_ = 1;
        buf_size = ((MpiEncTestData *)halCtx_)->frame_size;
​
        ret = hal_mpp_encode(halCtx_, dma_fd, (unsigned char *)buffer->GetVirAddress(), &buf_size);
        SerchIFps((unsigned char *)buffer->GetVirAddress(), buf_size, buffer);
​
        buffer->SetEsFrameSize(buf_size);
        clock_gettime(CLOCK_MONOTONIC, &ts);
        timestamp = ts.tv_nsec + ts.tv_sec * TIME_CONVERSION_NS_S;
        buffer->SetEsTimestamp(timestamp);
        CAMERA_LOGI("RKCodecNode::Yuv420ToH264 video capture on\n");
    } else {
        if (halCtx_ == nullptr) {
            CAMERA_LOGI("RKCodecNode::Yuv420ToH264 halCtx_ = %{public}p\n", halCtx_);
            return;
        }
        buf_size = ((MpiEncTestData *)halCtx_)->frame_size;
        ret = hal_mpp_encode(halCtx_, dma_fd, (unsigned char *)buffer->GetVirAddress(), &buf_size);
        SerchIFps((unsigned char *)buffer->GetVirAddress(), buf_size, buffer);
        buffer->SetEsFrameSize(buf_size);
        clock_gettime(CLOCK_MONOTONIC, &ts);
        timestamp = ts.tv_nsec + ts.tv_sec * TIME_CONVERSION_NS_S;
        buffer->SetEsTimestamp(timestamp);
    }
​
    CAMERA_LOGI("ForkNode::ForkBuffers H264 size = %{public}d ret = %{public}d timestamp = %{public}lld\n",
        buf_size, ret, timestamp);
}

8、针对当前usb camera不支持V4L2GetFmtDescs查询以及V4L2GetControls查询，去除对其探查结果的检查以及上报：

不支持信息日志打印如下：

 

代码修改如下：

drivers/peripheral/camera/hal/adapter/platform/v4l2/src/device_manager/v4l2_device_manager.cpp
void V4L2DeviceManager::UvcCallBack(const std::string hardwareName, std::vector<DeviceControl>& deviceControl,
    std::vector<DeviceFormat>& deviceFormat, bool uvcState)
{
    if (uvcState) {
        /*if (deviceControl.empty() || deviceFormat.empty()) {
            CAMERA_LOGI("V4L2DeviceManager::UvcCallBack %{public}s is empty", hardwareName.c_str());
            return;
        }*/
        //注释掉系统回调时对V4L2GetFmtDescs查询以及V4L2GetControls查询结果的判断
        CAMERA_LOGI("uvc plug in %{public}s begin", hardwareName.c_str());
        CameraId id = ReturnEnableCameraId("");
        CHECK_IF_EQUAL_RETURN_VOID(id, CAMERA_MAX);
​
        RetCode rc = GetManager(DM_M_SENSOR)->CreateController(DM_C_SENSOR, hardwareName);
        CHECK_IF_EQUAL_RETURN_VOID(rc, RC_ERROR);
​
        HardwareConfiguration hardware;
        hardware.cameraId = id;
        hardware.managerId = DM_M_SENSOR;
        hardware.controllerId = DM_C_SENSOR;
        hardware.hardwareName = hardwareName;
        hardwareList_.push_back(hardware);
        std::vector<float> physicalSize;
        for (auto iter = deviceFormat.cbegin(); iter != deviceFormat.cend(); iter++) {
            physicalSize.push_back((*iter).fmtdesc.width);
            physicalSize.push_back((*iter).fmtdesc.height);
        }
        std::shared_ptr<CameraMetadata> meta = std::make_shared<CameraMetadata>(ITEM_CAPACITY_SIZE,
            DATA_CAPACITY_SIZE);
        if (physicalSize.size() > 0) {
        //新增V4L2GetFmtDescs查询结果为空时不添加meta数据
            meta->addEntry(OHOS_SENSOR_INFO_PHYSICAL_SIZE, physicalSize.data(), physicalSize.size());
        }//新增V4L2GetFmtDescs查询结果为空时不添加meta数据
        CHECK_IF_PTR_NULL_RETURN_VOID(uvcCb_);
​
        uvcCb_(meta, uvcState, id);
        CAMERA_LOGI("uvc plug in %{public}s end", hardwareName.c_str());
    } else {
        CAMERA_LOGI("uvc plug out %{public}s begin", hardwareName.c_str());
        CameraId id = ReturnEnableCameraId(hardwareName);
        CHECK_IF_EQUAL_RETURN_VOID(id, CAMERA_MAX);
        CHECK_IF_PTR_NULL_RETURN_VOID(uvcCb_);
​
        for (auto iter = hardwareList_.cbegin(); iter != hardwareList_.cend(); iter++) {
            if ((*iter).hardwareName == hardwareName) {
                std::shared_ptr<CameraMetadata> meta =
                    std::make_shared<CameraMetadata>(ITEM_CAPACITY_SIZE, DATA_CAPACITY_SIZE);
                uvcCb_(meta, uvcState, id);
                hardwareList_.erase(iter);
                break;
            }
        }
        CAMERA_LOGI("uvc plug out %{public}s end", hardwareName.c_str());
    }
}

9、在CameraIdToHardware时匹配失败时返回空串，而不是nullptr；V4L2DeviceManager::UvcCallBack函数做如下修改：

drivers/peripheral/camera/hal/adapter/platform/v4l2/src/device_manager/v4l2_device_manager.cpp
......
std::string V4L2DeviceManager::CameraIdToHardware(CameraId cameraId, ManagerId managerId)
{
    for (auto iter = hardwareList_.cbegin(); iter != hardwareList_.cend(); iter++) {
        if ((*iter).managerId == managerId && (*iter).cameraId == cameraId) {
            return (*iter).hardwareName;
        }
    }
    return "";
    //匹配失败时返回空串，而不是空指针
    //return nullptr;
}
......
void V4L2DeviceManager::UvcCallBack(const std::string hardwareName, std::vector<DeviceControl>& deviceControl,
    std::vector<DeviceFormat>& deviceFormat, bool uvcState)
{
    if (uvcState) {
        /*if (deviceControl.empty() || deviceFormat.empty()) {
            CAMERA_LOGI("V4L2DeviceManager::UvcCallBack %{public}s is empty", hardwareName.c_str());
            return;
        }*/
        //注释掉系统回调时对V4L2GetFmtDescs查询以及V4L2GetControls查询结果的判断
        CAMERA_LOGI("uvc plug in %{public}s begin", hardwareName.c_str());
        //将空串改成hardwareName，如下：
        //CameraId id = ReturnEnableCameraId("");
        CameraId id = ReturnEnableCameraId(hardwareName);
        CHECK_IF_EQUAL_RETURN_VOID(id, CAMERA_MAX);
        //注释掉下面的部分----注释------start------
        /*RetCode rc = GetManager(DM_M_SENSOR)->CreateController(DM_C_SENSOR, hardwareName);
        CHECK_IF_EQUAL_RETURN_VOID(rc, RC_ERROR);
​
        HardwareConfiguration hardware;
        hardware.cameraId = id;
        hardware.managerId = DM_M_SENSOR;
        hardware.controllerId = DM_C_SENSOR;
        hardware.hardwareName = hardwareName;
        hardwareList_.push_back(hardware);*/
        //注释掉上面的部分----注释------end------
        
        std::vector<float> physicalSize;
        for (auto iter = deviceFormat.cbegin(); iter != deviceFormat.cend(); iter++) {
            physicalSize.push_back((*iter).fmtdesc.width);
            physicalSize.push_back((*iter).fmtdesc.height);
        }
        std::shared_ptr<CameraMetadata> meta = std::make_shared<CameraMetadata>(ITEM_CAPACITY_SIZE,
            DATA_CAPACITY_SIZE);
        if (physicalSize.size() > 0) {//新增V4L2GetFmtDescs查询结果为空时不添加meta数据
            meta->addEntry(OHOS_SENSOR_INFO_PHYSICAL_SIZE, physicalSize.data(), physicalSize.size());
        }//新增V4L2GetFmtDescs查询结果为空时不添加meta数据
        CHECK_IF_PTR_NULL_RETURN_VOID(uvcCb_);
​
        uvcCb_(meta, uvcState, id);
        CAMERA_LOGI("uvc plug in %{public}s end", hardwareName.c_str());
    } else {
        CAMERA_LOGI("uvc plug out %{public}s begin", hardwareName.c_str());
        CameraId id = ReturnEnableCameraId(hardwareName);
        CHECK_IF_EQUAL_RETURN_VOID(id, CAMERA_MAX);
        CHECK_IF_PTR_NULL_RETURN_VOID(uvcCb_);
​
        for (auto iter = hardwareList_.cbegin(); iter != hardwareList_.cend(); iter++) {
            if ((*iter).hardwareName == hardwareName) {
                std::shared_ptr<CameraMetadata> meta =
                    std::make_shared<CameraMetadata>(ITEM_CAPACITY_SIZE, DATA_CAPACITY_SIZE);
                uvcCb_(meta, uvcState, id);
                //hardwareList_.erase(iter);//注释掉这一行
                break;
            }
        }
        CAMERA_LOGI("uvc plug out %{public}s end", hardwareName.c_str());
    }
}

 

10、CameraHostImpl::Init函数和OnCameraStatus函数做如下修改：

drivers/peripheral/camera/hal/hdi_impl/src/camera_host/camera_host_impl.cpp
CamRetCode CameraHostImpl::Init()
{
......
    deviceManager->SetHotplugDevCallBack([this](const std::shared_ptr<CameraAbility> &meta,
        const bool &status, const CameraId &cameraId) {
            //CameraStatus cameraStatus = status ? AVAILABLE : UN_AVAILABLE;//注释掉这里，修改为下面的行
            CameraStatus cameraStatus = status ? AVAILABLE : DISAPPEAR;
            OnCameraStatus(cameraId, cameraStatus, meta);
        });
    return HDI::Camera::V1_0::NO_ERROR;
}
​
void CameraHostImpl::OnCameraStatus(CameraId cameraId,
    CameraStatus status, const std::shared_ptr<CameraAbility> ability)
{
    ......
    if (status == AVAILABLE) {
        // 先注释（见后面），再新增下面的代码
        RetCode rc = RC_OK;
        std::string logicalCameraId = config->ReturnLogicalCameraIdToString(physicalCameraId);
        if (logicalCameraId == "") {
            logicalCameraId = config->ReturnEnableLogicalCameraId();
            rc = config->AddCameraId(logicalCameraId, physicalCameraIds, ability);
        }
        if (rc == RC_OK && logicalCameraId.size() > 0) {
            CAMERA_LOGI("add physicalCameraIds %{public}d logicalCameraId %{public}s over",
                static_cast<int>(cameraId), logicalCameraId.c_str());
            if (cameraHostCallback_ != nullptr) {
                cameraHostCallback_->OnCameraStatus(logicalCameraId, status);
            }
        }//先新增前面这一段，然后注释掉后面的这一段
        // std::string logicalCameraId = config->ReturnEnableLogicalCameraId();
        // RetCode rc = config->AddCameraId(logicalCameraId, physicalCameraIds, ability);
        // if (rc == RC_OK && logicalCameraId.size() > 0) {
        //     CAMERA_LOGI("add physicalCameraIds %{public}d logicalCameraId %{public}s",
        //         static_cast<int>(cameraId), logicalCameraId.c_str());
        //     if (cameraHostCallback_ != nullptr) {
        //         cameraHostCallback_->OnCameraStatus(logicalCameraId, status);
        //     }
        // }
        std::shared_ptr<CameraDeviceImpl> cameraDevice =
            CameraDeviceImpl::CreateCameraDevice(logicalCameraId);
        if (cameraDevice != nullptr) {
            cameraDeviceMap_[logicalCameraId] = cameraDevice;
        }
    } else {
        std::string logicalCameraId =
            config->ReturnLogicalCameraIdToString(physicalCameraIds[0]);
        if (logicalCameraId.size() > 0) {
            CAMERA_LOGI("physicalCameraIds %{public}d logicalCameraId %{public}s",
                static_cast<int>(cameraId), logicalCameraId.c_str());
            if (cameraHostCallback_ != nullptr) {
                cameraHostCallback_->OnCameraStatus(logicalCameraId, status);
            }
        }
    }
}
​

11、HosFileFormat::V4L2GetCapability函数做如下修改：

drivers/peripheral/camera/hal/adapter/platform/v4l2/src/driver_adapter/src/v4l2_fileformat.cpp
RetCode HosFileFormat::V4L2GetCapability(int fd, const std::string& devName, std::string& cameraId)
{
    ......
    std::lock_guard<std::mutex> l(HosV4L2Dev::deviceFdLock_);
    std::string tmp("uvcvideo");//新增代码行
    std::string src = std::string(reinterpret_cast<char*>(cap.driver));//新增代码行
    if (src.compare(0, tmp.length(), tmp) != 0) {//新增代码行
        HosV4L2Dev::deviceMatch.insert(std::make_pair(std::string(reinterpret_cast<char*>(cap.driver)), devName));
​
        CAMERA_LOGD("v4l2 driver name = %{public}s\n", cap.driver);
        CAMERA_LOGD("v4l2 capabilities = 0x%{public}x\n", cap.capabilities);
        CAMERA_LOGD("v4l2 card: %{public}s\n", cap.card);
        CAMERA_LOGD("v4l2 bus info: %{public}s\n", cap.bus_info);
    }//新增代码行
​
    return RC_OK;
}

 

12、HosV4L2Dev::start(const std::string& cameraID)函数做如下修改：

drivers/peripheral/camera/hal/adapter/platform/v4l2/src/driver_adapter/src/v4l2_dev.cpp
......
fd = GetCurrentFd(cameraID);
    if (fd < 0) {
        CAMERA_LOGE("error: ReqBuffers: GetCurrentFd error\n");
        return RC_ERROR;
    }
​
    rc = myStreams_->V4L2StreamOff(fd);
    if (rc == RC_ERROR) {
        CAMERA_LOGE("error: StartStream: V4L2StreamOn error\n");
        //return RC_ERROR;//注释掉这一行
    }
​
    EraseEpoll(fd);
​
    if (streamNumber_ == 0) {
        close(epollFd_);
        delete streamThread_;
        streamThread_ = nullptr;
    }
​
    return RC_OK;
}

 

13、HosV4L2UVC::V4L2UvcGetCap(const std::string v4l2Device, struct v4l2_capability& cap)函数做如下修改：

drivers/peripheral/camera/hal/adapter/platform/v4l2/src/driver_adapter/src/v4l2_uvc.cpp
......
char *devName = nullptr;
    char absPath[PATH_MAX] = {0};
​
    devName = realpath(v4l2Device.c_str(), absPath);
    if (devName == nullptr) {
        CAMERA_LOGE("UVC:V4L2UvcGetCap realpath error v4l2Device == %{public}s\n", v4l2Device.c_str());
        return RC_ERROR;
    }
​
    fd = open(devName, O_RDWR | O_NONBLOCK, 0);
    //free(devName);注释掉这一行，因为devName指向的地址空间实际是absPath数组，不需要释放
    if (fd < 0) {
        CAMERA_LOGE("UVC:ERROR opening V4L2 interface for %{public}s\n", v4l2Device.c_str());
        return RC_ERROR;
    }
......

14、BufferAdapter::PixelFormatToCameraFormat和CameraFormatToPixelFormat函数,做如下修改：

drivers/peripheral/camera/hal/buffer_manager/src/buffer_adapter/standard/buffer_adapter.cpp
uint32_t BufferAdapter::PixelFormatToCameraFormat(const PixelFormat format)
{
......
        case PIXEL_FMT_YCRCB_420_SP:
            cameraFormat = CAMERA_FORMAT_YCRCB_420_SP;
            break;
        case PIXEL_FMT_YCBCR_422_P:
            cameraFormat = CAMERA_FORMAT_YCBCR_422_P;
            break;
        case PIXEL_FMT_YCRCB_422_P:
            //cameraFormat = CAMERA_FORMAT_YCRCB_422_P;//上层很多模块以及测试demo都是使用的CAMERA_FORMAT_YCRCB_420_SP枚举定义
            cameraFormat = CAMERA_FORMAT_YCRCB_420_SP;
            break;
        case PIXEL_FMT_YCBCR_420_P:
            cameraFormat = CAMERA_FORMAT_YCBCR_420_P;
            break;
......
}
​
PixelFormat BufferAdapter::CameraFormatToPixelFormat(const uint32_t cameraFormat)
{
......
        case CAMERA_FORMAT_YCBCR_420_SP:
            format = PIXEL_FMT_YCBCR_420_SP;
            break;
        case CAMERA_FORMAT_YCRCB_420_SP:
            //format = PIXEL_FMT_YCRCB_420_SP;//上层很多模块以及测试demo都是使用的PIXEL_FMT_YCRCB_420_SP枚举定义
            format = PIXEL_FMT_YCRCB_422_P;
            break;
        case CAMERA_FORMAT_YCBCR_422_P:
            format = PIXEL_FMT_YCBCR_422_P;
            break;
......
}

15、ForkNode::DeliverBuffer函数,做如下修改：

drivers/peripheral/camera/hal/pipeline_core/nodes/src/fork_node/fork_node.cpp
void ForkNode::DeliverBuffer(std::shared_ptr<IBuffer>& buffer)
{
......
            std::shared_ptr<IBuffer> forkBuffer = bufferPool_->AcquireBuffer(0);
            if (forkBuffer != nullptr) {
                if (memcpy_s(forkBuffer->GetVirAddress(), forkBuffer->GetSize(),
                    buffer->GetVirAddress(), (buffer->GetWidth()*buffer->GetHeight()*2)) != 0) {
                    将尺寸*3/2改为*2
                    //buffer->GetVirAddress(), (buffer->GetWidth()*buffer->GetHeight()*3/2)) != 0) {
                    forkBuffer->SetBufferStatus(CAMERA_BUFFER_STATUS_INVALID);
                    CAMERA_LOGW("memcpy_s failed.");
                }
......
}

16、enum CameraStatus ,做如下修改，增加APPER和DISAPPER枚举类型：

drivers/interface/camera/v1_0/Types.idl
......
/**
 * @brief Enumerates camera device statuses.
 */
enum CameraStatus {
    /**
     * The new camera device is APPER.
     */
    APPEAR = 0,
​
    /**
     * The camera device is DISAPPEAR.
     */
    DISAPPEAR = 1,
    /**
     * The camera device is available.
     */
    AVAILABLE = 2,
​
    /**
     * The camera device is not in position or is unavailable.
     */
    UN_AVAILABLE = 3,
};
......

 

17、g_cameraToPixelFormat映射,做如下修改：

foundation/multimedia/camera_framework/services/camera_service/src/camera_util.cpp
std::unordered_map<int32_t, int32_t> g_cameraToPixelFormat = {
    {OHOS_CAMERA_FORMAT_RGBA_8888, PIXEL_FMT_RGBA_8888},
    {OHOS_CAMERA_FORMAT_YCBCR_420_888, PIXEL_FMT_YCBCR_420_SP},
    {OHOS_CAMERA_FORMAT_YCRCB_420_SP, PIXEL_FMT_YCRCB_422_P},//更换为PIXEL_FMT_YCRCB_422_P
    {OHOS_CAMERA_FORMAT_JPEG, PIXEL_FMT_YCRCB_422_P},//更换为PIXEL_FMT_YCRCB_422_P
    //{OHOS_CAMERA_FORMAT_YCRCB_420_SP, PIXEL_FMT_YCRCB_420_SP},
    //{OHOS_CAMERA_FORMAT_JPEG, PIXEL_FMT_YCRCB_420_SP},
};

 

18、HCameraHostManager::CameraHostInfo::OnCameraStatus函数,做如下修改：

int32_t HCameraHostManager::CameraHostInfo::OnCameraStatus(const std::string& cameraId,
                                                           HDI::Camera::V1_0::CameraStatus status)
{
    if ((cameraHostManager_ == nullptr) || (cameraHostManager_->statusCallback_ == nullptr)) {
        MEDIA_WARNING_LOG("CameraHostInfo::OnCameraStatus for %{public}s with status %{public}d "
                          "failed due to no callback",
                          cameraId.c_str(), status);
        return CAMERA_UNKNOWN_ERROR;
    }
    CameraStatus svcStatus = CAMERA_STATUS_UNAVAILABLE;
    switch (status) {
        case UN_AVAILABLE: {
            MEDIA_INFO_LOG("CameraHostInfo::OnCameraStatus, camera %{public}s unavailable", cameraId.c_str());
            svcStatus = CAMERA_STATUS_UNAVAILABLE;
            break;
        }
        case AVAILABLE: {
            MEDIA_INFO_LOG("CameraHostInfo::OnCameraStatus, camera %{public}s available", cameraId.c_str());
            svcStatus = CAMERA_STATUS_AVAILABLE;
            AddDevice(cameraId);
            break;
        }
        case APPEAR: {//增加APPEAR分支
            MEDIA_INFO_LOG("CameraHostInfo::OnCameraStatus, camera %{public}s APPEAR", cameraId.c_str());
            svcStatus = CAMERA_STATUS_APPEAR;
            break;
        }
        case DISAPPEAR: {//增加DISAPPEAR分支
            MEDIA_INFO_LOG("CameraHostInfo::OnCameraStatus, camera %{public}s DISAPPEAR", cameraId.c_str());
            svcStatus = CAMERA_STATUS_DISAPPEAR;
            break;
        }
        default:
            MEDIA_ERR_LOG("Unknown camera status: %{public}d", status);
            return CAMERA_UNKNOWN_ERROR;
    }
    cameraHostManager_->statusCallback_->OnCameraStatus(cameraId, svcStatus);
    return CAMERA_OK;
}

19、框架层添加USB camera移除信息上报以后，Camera App端需要做移除信息的接收显示以及在检测到USB camera被移除以后做camera的close操作，才能确保底层系统资源被释放。这样USB camera再次插上时，才能再次出图，如此则在最小适配改动下，支持了热插拔：

注：做USB camera实际应用时需要监听cameraStatus消息( 下图为伪代码，app需要自行开发)。如果初期USB camera调试，可以先不改camera app。camera的close操作，可以通过在拔除USB camera后，手动退出camera APP，来达到相同的调试效果。 附件提供了调试对应带cameraStatus消息监听的camera hap包。

监听cameraStatus消息，USB camera移除消息，收到消息后先close相机，然后再提示用户USB camera被移除。这样即使用户不点击确定，再次插上USB camera时，因为旧底层的设备已经被关闭，资源被释放。新增的USB Camera也不会因旧资源冲突，出现设备节点的增长错位。设备节点的增长错位会导致新插入的USB camera不出图。