手机拍照进化论：为什么需要图像算法？

<p><span>人类进入智能手机时代后，摄影方式也发生了巨大变化：用来拍照的不再只是镜头和传感器了，背后还加载着一系列的图像算法，它可以用更快的速度处理图像，并获得更好的拍摄效果。</span></p>

<p><span>好作品只能依赖专业摄影师的时代过去了，图像算法引入到手机拍照应用后，</span><span>让人人拍出好照片成为可能。</span></p>

<p><span>紫光展锐拥有自主研发的图像算法团队，而且这些算法已普遍应用在展锐的芯片平台里。目前，展锐的图像算法团队已经超过了</span><span>200人，</span><span>分布在</span><span>美国、印度、中国台湾、上海和北京等多个国家和地区。</span></p>

<p><span>更好的手机拍照效果需要图像算法的加持，为了让大家有更好的了解，接下来的系列，我们准备了几篇科普、视频和图说，一起来看看吧~</span><span>~~</span></p>

<p><span>这事得从人类对色彩的感知能力说起</span><span>…</span></p>

<p><span>人类</span><span>对色彩</span><span>的感知能力是</span><span>漫长</span><span>进化过程中产生的一项重要能力，它使得我们</span><span>眼中</span><span>的</span><span>世界</span><span>五彩斑斓。</span><span>颜色是怎么呈现出来的呢？它</span><span>是</span><span>光</span><span>线通过</span><span>眼睛成像到</span><span>视网膜上，</span><span>由视网膜</span><span>上的视</span><span>细胞</span><span>感知</span><span>光</span><span>信号</span><span>的</span><span>强弱</span><span>，视觉</span><span>神经对信号的传播，</span><span>再由</span><span>大脑处理所产生的</span><span>视觉</span><span>感受</span><span>。颜色的</span><span>产生极其复杂，</span><span>人类所获得</span><span>的色彩感受不但</span><span>取决于</span><span>光的物理特性，还</span><span>受</span><span>心理</span><span>等</span><span>因素</span><span>影响。</span></p>

<p><span>首先</span><span>，我们先来了解一下颜色的物理特性。肉眼所见到的光线</span><span>（一般波长范围是</span><span>380-780nm</span><span>，</span><span>见图1），是</span><span>电磁波谱中人眼可以感知到的部分</span><span>，不同波长的</span><span>可见光</span><span>表现为不同的颜色，对色彩的</span><span>辨别</span><span>是肉眼受到</span><span>可见</span><span>的电磁波辐射能刺激后引</span><span>发</span><span>的视觉</span><a href="https://baike.baidu.com/item/%E7%A5%9E%E7%BB%8F/17272"><span>神经</span><…。</span></p>

<p><img alt="图1：可见光波长及其对应的颜色" data-entity-type="file" data-entity-uuid="63d76f7e-832e-4f4e-9c8b-adebedee652a" src="http://new.eetrend.com/files/2020-11/wen_zhang_/100059266-114232-1.jpg&…; /></p>

<p><em><span>图</span><span>1</span><span>：可见</span><span>光</span><span>波长及其</span><span>对应的</span><span>颜色</span></em></p>

<p><span>人眼</span><span>中对光敏感的细胞分为两类</span><span>：</span><span>视</span><span>杆</span><span>细胞</span><span>和视锥细胞</span><span>。</span><span>其中</span><span>，</span><span>视杆</span><span>细胞对</span><span>弱</span><span>光线</span><span>最</span><span>敏感，</span><span>光谱</span><span>响应范围覆盖整个可见光区域，在500nm附近达到</span><span>相应</span><span>的峰值</span><span>，</span><span>但无法区分不同的波段的</span><span>光线</span><span>，因此</span><span>无法</span><span>产生色觉。在非常低的光照水平下，</span><span>这样</span><span>的视觉</span><span>叫做</span><a href="https://wiki.tw.wjbk.site/w/index.php?title=%E6%9A%97%E8%A7%86%E8%A7%89…;暗视觉</span></a><span>，</span><span>暗视觉仅视</span><span>杆</span><span>细胞</span><span>在</span><span>工作</span><span>。</span><span>所以</span><span>在</span><span>黑暗的夜晚，人们能看到物体，却分辨不出</span><span>物体的</span><span>颜色。</span></p>

<p><span>在</span><span>高光照</span><span>水平</span><span>下，</span><span>主要工作</span><span>的</span><span>是视锥细胞，</span><span>有色</span><span>觉</span><span>，这样的视觉叫明视觉。视锥细胞</span><span>分为</span><span>三类</span><span>：</span><span>感红细胞</span><span>(</span><span>L</span><span>)</span><span><span>&nbsp;</span>、</span><span>感</span><span>绿细胞</span><span>(</span><span>M)和感蓝</span><span>细胞(</span><span>S)</span><span>。对</span><span>同一波长的光线，不同的视锥细胞有不同的</span><span>响应</span><span>能力</span><span>（见</span><span>图2）</span><span>，因此</span><span>视锥细胞具有</span><span>颜色</span><span>的辨别能力。</span></p>

<p><img alt="图2：三种视锥细胞对光的响应能力" data-entity-type="file" data-entity-uuid="efd0bde2-31c2-4c1b-a667-290ccf70cd7f" src="http://new.eetrend.com/files/2020-11/wen_zhang_/100059266-114233-2.jpg&…; /></p>

<p><em><span>图</span><span>2</span><span>：三种</span><span>视锥细胞对光的响应能力</span></em></p>

<p><span>光源发射某一光谱</span><span>分布的电磁波，经过物体的</span><span>反射</span><span>或者</span><span>透射</span><span>进入人眼，</span><span>再</span><span>通过视锥细胞</span><span>对</span><span>光线的探测，</span><span>这</span><span>整个过程涵盖了</span><span>颜色</span><span>的物理属性</span><span>。颜色</span><span>同时受到人们的心理、现实经验等因素的影响</span><span>，</span><span>某</span><span>个</span><span>场景的光在视</span><span>细胞</span><span>产生的信号并不等于人们对这个场景的感受。例如</span><span>，</span><span>一张白纸</span><span>不管</span><span>在红色还是绿色灯光下，</span><span>人眼</span><span>看起来都</span><span>会</span><span>觉得这是一张白纸，这来源于</span><span>大脑</span><span>的</span><span>经验</span><span>处理，这种现象</span><span>叫做</span><span>颜色恒常</span><span>性。</span></p>

<p><span>可以说，你看到的景色都是经过大脑处理加工过的，眼见并不一定为实哦~</span></p>

<p><span>图像算法最大的用途是可以修复照片，</span><span>使拍摄到</span><span>的照片</span><span>获得</span><span>真实的色彩效果，或者满足</span><span>用户</span><span>的某些色彩偏好。</span></p>

<p><span>【</span><span>自动白平衡</span><span>】</span></p>

<p><span>在展锐的ISP（</span><span>Image Signal Processing</span><span>）系统里</span><span>，主要有自动白平衡</span><span>（AWB）和</span><span>颜色校正</span><span>（CC</span><span>M</span><span>）</span><span>模块来处理色彩。</span></p>

<p><span>先看</span><span>自动白平衡</span><span>，ISP系统需要根据图像</span><span>的</span><span>统计信息，计算出图像R、B</span><span><span>&nbsp;</span></span><span>channel的增益，以实现</span><span>自动白平衡</span><span>功能，以求在各种复杂场景下均能准确还原物体本来的颜色。因为人类视觉系统具有颜色恒常性的特点，对事</span><span>物的观察可以不受光源的影响，但图像传感器不具有这个特点，因此在不同光线下拍摄的图像，会呈现出不同的颜色。比如，在晴朗的天空下拍摄的图像可能偏蓝，而在烛光下拍摄的物体可能偏红。</span></p>

<p><span>自动白平衡功能，就是模拟了人类视觉系统的颜色恒常性特点，来消除光源对图像成像的不良影响。</span></p>

<p><img alt="图3 ：左边是自动白平衡功能关闭，右边是自动白平衡开启" data-entity-type="file" data-entity-uuid="d311defe-0321-456a-bf74-ab30296cbe6b" src="http://new.eetrend.com/files/2020-11/wen_zhang_/100059266-114234-3.jpg&…; /></p>

<p><em><span>图</span><span>3<span>&nbsp;</span></span><span>：</span><span>左边是自动白平衡</span><span>功能关闭</span><span>，右边是自动白平衡</span><span>开启</span></em></p>

<p><span>紫光展锐</span><span>的</span><span>自动</span><span>白</span><span>平衡</span><span>算法</span><span>具有以下</span><span>特点</span><span>：</span></p>

<p><span>效果稳定性好：</span><span>所谓效果稳定性，指的是</span><span>拍照</span><span>场景发生细微改变时，图像色彩效果不会出现明显</span><span>跳变</span><span>。</span></p>

<p><span>准确度高：</span><span>展锐的</span><span>算法在处理绝大部分场景时</span><span>，</span><span>都能得到</span><span>符合</span><span>预期的结果</span><span>，尤其</span><span>是用户关心的蓝天、绿植、</span><span>人物</span><span>等等重点场景</span><span>和混合</span><span>光源</span><span>、纯色</span><span>、</span><span>夜景等难点</span><span>场景。</span></p>

<p><span>适用</span><span>范围广</span><span>、</span><span>灵活性好</span><span>：</span><span>展锐的算法</span><span>既可以运行在</span><span>入门级相机</span><span>设备</span><span>上（如30万</span><span>像素的手表</span><span>项目</span><span>）上</span><span>，</span><span>也可以运行在中高端相机设备</span><span>上（如3200万</span><span>像素</span><span>以上</span><span>的手机</span><span>项目</span><span>）</span><span>，还可以</span><span>应用到车载记录</span><span>等物联网</span><span>领域</span><span>中</span><span>。</span><span>而且，</span><span>不同</span><span>产品</span><span>，</span><span>展锐的</span><span>自动白平衡算法</span><span>可以</span><span>实现灵活配置</span><span>，实现算法</span><span>效果</span><span>、算法复杂度</span><span>、内存</span><span>消耗等多样化</span><span>的</span><span>分级</span><span>需求。</span></p>

<p><span>【</span><span>颜色校正</span><span>】</span></p>

<p><span>相机实际</span><span>是在模拟人眼的彩色成像</span><span>来</span><span>拍图，</span><span>由于相机</span><span>传感器</span><span>的</span><span>RGB三通道的光谱响应与人眼</span><span>中</span><span>视</span><span>细胞</span><span>的</span><span>光谱</span><span>响应</span><span>（见</span><span>图2）不同，导致</span><span>其</span><span>拍摄的图片与人眼看到的有差别</span><span>，</span><span>因此需要进行颜色校正</span><span>（Color Correction Matrix，C</span><span>CM</span><span>）</span><span>。</span></p>

<p><span>颜色校正矩阵一般通过标定得到，利用图像传感器拍摄得到的图像与标准图像相比较，计算得到校正矩阵（即</span><span>CCM</span><span>矩阵</span><span>）</span><span>。利用C</span><span>CM</span><span>矩阵对图像的</span><span>各</span><span>像素点</span><span>的</span><span>进行验收校正，以获得最接近于物体真实颜色的图像。</span></p>

<p><span>一般情况下，在颜色校正过程中，都会伴随有颜色饱和度的调整。颜色饱和度是指色彩的纯度，其色彩的纯度越高，图像表现越鲜明；纯度越低，表现就越黯淡。</span></p>

<p><img alt="图4 ：左边是CCM关闭，右边是CCM开启" data-entity-type="file" data-entity-uuid="0a75f271-3c70-41c2-b9f9-3315f3a40616" src="http://new.eetrend.com/files/2020-11/wen_zhang_/100059266-114235-4.jpg&…; /></p>

<p><em><span>图</span><span>4<span>&nbsp;</span></span><span>：</span><span>左边是CCM</span><span>关闭</span><span>，右边是CCM</span><span>开启</span></em></p>

<p><img alt="图5：展锐CCM模块的色彩还原效果评测 （图中18个小圆点和小方点越接近，则表示色彩还原性越好）" data-entity-type="file" data-entity-uuid="1ca22c1f-6ca4-4210-a9bb-cf4cb3936934" src="http://new.eetrend.com/files/2020-11/wen_zhang_/100059266-114236-5.jpg&…; /></p>

<p><em><span>图</span><span>5</span><span>：展锐CCM模块的</span><span>色彩还原效果</span><span>评测</span></em></p>

<p><em><span>（图</span><span>中</span><span>18个小圆点和</span><span>小方点</span><span>越接近</span><span>，</span><span>则表示</span><span>色彩</span><span>还原性</span><span>越好</span><span>）</span></em></p>

<p><span>随着机器学习等</span><span>高新技术的发展</span><span>，</span><span>以及</span><span>用户</span><span>对手机</span><span>拍照功能</span><span>的</span><span>需求日益</span><span>苛刻，</span><span>色彩</span><span>还原技术</span><span>也会</span><span>面临</span><span>一些</span><span>新的机遇</span><span>和</span><span>挑战，例如</span><span>如何</span><span>利用</span><span>AI技术提升</span><span>色彩还原效果，如何</span><span>在</span><span>多摄像头</span><span>手机</span><span>上</span><span>实现更加一致</span><span>的</span><span>色彩还原。</span><span>这些</span><span>附加技术</span><span>将</span><span>给用户带来</span><span>更好的</span><span>手机</span><span>拍照体验，</span><span>进而持续</span><span>提升</span><span>公司</span><span>产品的竞争力。</span></p>