薛定谔——赋予生命科学以革命性契机的理论物理学家
量子力学，在生命的世界里
quantum 量子 机制 光合 物理 遗传 DNA 嗅觉
Johnjoe McFadden 发表于 2014-11-29 11:11
图片来源：Takashi Hososhima/Flickr但没有人真的相信这只猫可以既死了又活着。原子这样的基本粒子有奇怪的量子特性（比如同时存在两种状态，同时处于两个位置，穿过本应无法透过的屏障等等），而猫这样常见的经典物体却不会有，两者之间存在极其重大的区别。为什么呢？简而言之，这些奇怪的量子特性太脆弱了。
薛定谔的猫不太可能真的既死又活——但它体内会不会有别的量子叠加态呢？图片来源：egotailor.com然而，薛定谔1944年的《生命是什么》（What Is Life）一书中却写道，生命中一些最为基础的砖石，必定会像肉眼看不到的放射性原子一样，是一种量子实体，具有反直觉的特征。实际上薛定谔认为，生命和非生命之所以不同，正是因为生命存在于量子世界和经典世界之间的中间地带——我们可以称之为“量子边界”。
“有序中诞生有序”
薛定谔的观点是基于以下这些看起来矛盾的事实。尽管经典定律——从牛顿力学定律、热力学定律到电磁学定律——看起来都极其有序，但实际上，它们都基于无序。设想有一个气球，里面充满了数万亿进行无序运动的气体分子，不断撞击着彼此和气球内壁。但是，当你把它们的运动加和再平均后，你就得到了气体定律，而用这一定律可以准确地推导出气球受热后会膨胀。薛定谔将这种定律称作“无序中诞生的有序”，以此来说明宏观上的规律，其实依赖于粒子水平上的混乱和不可预测。那这和生命有什么关系呢？其实，薛定谔对遗传非常感兴趣。1944年，当时距沃森和克里克揭示DNA分子的结构尚有10年时间，基因的物理本质还是一个谜。即使如此，人们已经知道了基因要想一代一代传下去，就必然具有极高的保真度：出错的概率小于十亿分之一。这是个谜，因为当时已经知道基因非常小——薛定谔认为，基因小到不能依赖“无序中诞生的有序”定律来保证其复制的准确性。他提出，这一过程必定有一个“更复杂的有机分子”参与其中，在这个大分子中，“每个原子、每组原子，都扮演着各自的角色”。
薛定谔将这些新结构称为“非周期晶体”。他声称它们一定是遵守量子规律而非经典法则，并进一步提出基因突变可能是晶体内的量子跃迁导致的。他接着提出，生命的许多特征或许建立在一个新的物理原则上。在非生命的世界里，如我们所知，宏观规律通常来自无序的分子：有序来自无序。但也许——薛定谔说——生命世界里的宏观规律反映了另一些东西：量子级别的神秘规律。他把这种推测称为“有序来自有序”。
他是对的吗？
十年之后，沃森和克里克发现了双螺旋结构。基因的成分原来是单分子DNA——一条长链分子，上面像珠子一样穿满了碱基。它完全就是一个非周期晶体，只是没叫这个名字而已。并且，就像薛定谔预测的，“每个原子团”的的确确都扮演着“一个独立角色”，甚至单个质子都发挥它们的量子特性，确定各自的碱基。整个科学史上恐怕找不出几个比这更有先见之明的预测了。你眼睛的颜色，你鼻子的形状，你的性格、智力甚至患病的可能，都在量子级别编码。
非周期晶体本身后来成为了另一个同样迷人的领域。图片来源：www.complexphotonics.org从量子嗅觉到量子导航
让我们从几个相对边缘的例子说起——比如嗅觉。关于嗅觉的传统理论认为，气味分子会被味觉受体探测到，靠的是鼻子内一种钥匙-锁结构：气味分子与受体的空隙结合，然后触发反应，就像钥匙转动了锁。这是一种令人愉快、非常直观的学说，但是它解释不了某些现象——例如，外形相似的分子经常会闻起来不一样，反之亦然。经过修正的学说认为，感受器也许是对分子振动做出回应。在1996年这个想法在量子学层面得到了进一步的解释——生物物理学家卢卡•都灵（Luca Turin）提出振动可能会促进电子的量子隧道效应。打开嗅觉的“锁”。嗅觉的量子理论也许听起来很奇怪，但最近出现了支持的证据：果蝇可以分辨形状完全相同、只是用了同一元素不同同位素的气味分子，这用量子力学之外的理论很难解释清楚。或者考虑一下这个问题：我们已知一些鸟类和其他动物会通过感知地球上非常微弱的磁场来导航，但它们是怎么办到的，一直是个谜。很难想象到如此微弱的磁场如何在动物体内产生一个信号。在另一个关于欧亚鸲的研究中出现了更深层次的问题，这种鸟的导航系统依赖光线，并且不同于常规的指南针，它探测的不是磁感线的朝向，而是磁感线相对于地表的角度。没有人知道为什么。
直到20世纪70年代，德国化学家克劳斯•舒特恩（Klaus Schulten）发现一些化学反应产生的粒子对会保持连接状态，靠的是一种特殊的量子属性——量子缠结。量子缠结允许远距离的粒子维持即时通讯，无论它们之间有多远，即便被扔在银河系的两端，它们之间仍然能难以理解地相互关联。量子缠结是如此诡异以至于提出黑洞和时空扭曲理论的阿尔伯特•爱因斯坦（Albert Einstein）本人说这是“鬼魅似的远距作用”。但数以百计的实验证明这是真实的。
舒特恩发现，缠结的粒子对会对磁场的强度和方向极其敏感。他认为神秘的鸟类导航也许用到了粒子的量子纠缠。几乎没有人认同这个观点，但在2000年时，舒特恩和他的学生索斯藤•丽兹（Thorsten Ritz）写了一篇很有影响力的文章，这篇文章展示了在鸟的眼睛中，光是如何影响量子缠结导航的。在2004年，丽兹与著名鸟类学家沃尔夫冈和罗斯维塔•威尔科奇夫妇合作，他们找到了能令人信服的实验证据，证明欧亚鸲每年在全球范围内迁徙时，的确存在爱因斯坦所说的“鬼魅”作用。
毫无疑问，导航和嗅觉非常重要，但这些对地球上的生命来说可能还谈不上核心需求。那么我们来看看更主要的是什么。
会传送的电子和长眼睛的光
比方说酶。它们是生命世界的老黄牛，能够加速化学反应，在几秒内就完成要花数千年才能完成的过程。酶往往能让反应加快几万亿倍，但它是怎么做到这一点的，一直是个谜。不过现在，加州伯克利大学的朱迪思•克兰曼（Judith Klinman）和曼彻斯顿大学的奈杰尔•斯克鲁顿（Nigel Scrutton）等人发现，酶有一个神奇的量子小窍门——隧道效应。简单来说，酶在生物化学反应中促进了这样个一过程：电子和质子从生化分子的某处消失，同一瞬间在另一个地方出现，而不必经过中间的任何地方——也就是某种意义上的“传送”。这都是非常基本的东西。这个星球上每个生物的每个细胞中的每一个生物分子，都是酶创造的。酶比任何其他成分（哪怕DNA，毕竟有些细胞没有DNA也能活）都更有资格称为生命的必备成分。而它们靠浸入量子世界来帮助我们存活下去。
我们还可以把论证再往前推一步。光合作用是地球上最重要的生化反应。它负责将光，空气，水和少量矿物质转变成草，树木，粮食以及以植物或食草动物为食的我们。起初是由叶绿素分子捕获光能。该光能被转化为电能，然后这些电能将被输送到被称为反应中心的生化工厂，在那里它们被用来固定二氧化碳并将其转化成植物物质。长期以来，这种能源运输的过程让研究者们着迷，因为它可以如此高效——接近100％ 。绿叶运输能量的过程是如何做到比我们最先进的技术还要好的？
在加州大学伯克利分校，格雷厄姆•佛莱明（Graham Fleming）的实验室已经利用“飞秒光谱技术”对光合作用的效率问题进行了十多年的研究。从本质上说，这个研究小组就是在极短的时间内往光合作用复合物上照射激光，以找出光子抵达光合反应中心的路径。早在2007年，这个小组就研究了细菌中的FMO复合物。在这个复合物中，光子的能量需要通过一簇叶绿素分子。人们曾认为在这个过程中，光子会如同带电粒子一样从一个叶绿素分子跳到另一个叶绿素分子上，就好比薛定谔的猫在横渡溪流时可能会从一块石头跳到另一块上一样。但这种解释并不完全说得通。光子可没有方向感，大多数光能应该会漫无目的地往错误的方向传递，最终一头栽到“溪水”里。可是，在植物和能进行光合作用的细菌中，几乎全部光能都传到了光合反应中心。
当研究小组向FMO系统发射激光时，他们观察到了古怪的光回波——像是打着节奏一般的波。这些“量子鼓点”意味着光子的能量不是通过单一路径传入光合系统的，而是利用量子相干性同时从所有可能的路径进行传递。想象一下，薛定谔的猫咪在面对溪流时，不知怎的将自己分成了众多完全一样的、具有量子相干性的小猫咪。它们从所有可能的路线跳过一个个叶绿素砾石，来探寻最快捷的路线。现在，量子鼓点已经在多种不同的光系统中被侦测到，像菠菜这样的普通植物的光系统也不例外。这样看来，为了让我们有吃的，生物界最重要的反应都在动用量子世界的资源了呀。
如果对你来说，这样都还不够的话，我们最后来看看演化机制本身吧。薛定谔认为突变可能与一种量子跃迁有关。在沃森和克里克那篇经典的DNA文章中，他们提出基因突变可能牵扯到核苷酸碱基的“互变异构”——互变异构过程被认为与量子隧穿效应有关。在1999年，吉姆•艾尔-卡里利（Jim Al-Khalili）和我觉得质子隧穿可能解释一种特别的突变类型——所谓的“适应性突变”。当这种突变能为个体带来好处时，这种突变似乎就会更加频繁地发生。我们当时的论文完全是理论性的，但我们现在正在试图为DNA中的质子隧穿找到实验证据。所以，请拭目以待。
尽管有了这么多量子解释来阐述令人费解的生命现象，我们却发现自己陷入了更深的谜团中。量子相干性是个极度脆弱的现象，依赖于粒子波的步调一致。为了保持量子相干性，物理学家们通常不得不将系统置于几乎完全真空的状态，并且将系统降温至接近绝对零度，以停止任何热驱动的分子运动。分子振动是量子相干性的死敌。
那如果是那样的话，为什么生物能够设法在足够长的时间里保持自己的分子秩序，以在温暖潮湿的细胞环境中也能上演量子戏法？这仍旧是个深奥的谜题。最近有研究提供了一项诱人提示：生物不但没有试图避免那分子的“风暴”，反而接受了它们。生物就像是利用疾风狂潮来保持船体直立着驶往正确航向的船长一样。正如薛定谔所预言的那样，生命是沿着量子边界——经典世界与量子世界间的狭窄“溪流”——在悠然航行。（编辑：Ent）
薛定谔（Erwin Schrdinger, 1887～1961）,奥地利理论物理学家。1906年至1910年在维也纳大学物理系学习。1910年获博士学位后，在维也纳大学第二物理研究所工作。1921年至1927年在瑞士苏黎士大学任数学物理教授，1927年接替普朗克到柏林大学担任理论物理学教授。出于对纳粹政权的愤慨，1933年移居英国牛津。1939年转到爱尔兰，在都柏林高级研究所工作了17年，直到1956年返回奥地利。1961年元月在奥地利阿尔卑巴赫山村病逝。
薛定谔最杰出的贡献是在1925年底至1926年初提出了用波动力学方程来处理电子运动问题，得到了与实验数据相符合的结果，这一方程以后被称为薛定谔方程。他还证明了波动力学和矩阵力学在数学上是等价的，是量子力学的两种形式。为此，薛定谔荣获1933年的诺贝尔物理学奖。以后，他致力于研究有关波动力学的应用和统计诠释，以及广义相对论和统一场论问题。
1944年，薛定谔发表《生命是什么》一书。在书中，他试图用热力学、量子力学等理论来解释生命现象，引入了负熵、遗传密码、量子跃迁式的突变等概念，这些概念至今仍有着广泛的影响。他是公认的分子生物学的先驱。
* V; p# _2 L. c4 R8 |2 m! F q
物理学家们如何看待生命的本质
生命究竟是什么？生命运动有没有规律？多少年来，人们都在积极思考这些问题。20世纪以来，随着物理学和生物学的发展，物理学不可避免地向生物学渗透。早在20世纪初，美国遗传学家摩尔根的一位小有名气的同学洛布就提出，生命运动的规律同一般的物理学和化学的规律是完全一致的。所以要理解生命现象，就必须用物理学、化学的观点，用实验、定量的方法来研究生物学。
物理学家们也饶有兴趣地议论生物学的问题。30年代，丹麦著名的物理学家、量子论的奠基人玻尔对于生命问题发表了非常有意义的见解。1932年他在题为“生命和光”的演讲中指出，试图把有机体简单地还原为化学的相互作用来回答“生命是什么？”的问题，就如同试图画出每个电子的位置来描述原子一样困难。简单地将生物体分解为其组成部分的化学系统，那样机体就不能活了，就成为另外的不同系统了。在生物学里承认生命的存在是一个给定的基本事实。旧的概念，即机械论对此不可能作出满意的结论。生物学也必须像物理学那样，当它运用新的概念和新的研究方法时，就能上升到新的认识水平。玻尔天才地预示了物理学与生物学或许也是互补的，这两者相结合将对生命现象作出满意的解释。
玻尔的思想启发了他的学生德尔布吕克。德尔布吕克20年代到玻尔的理论物理所工作过两年。30年代他到美国的加州理工学院，与摩尔根等遗传学家交往甚密，兴趣转到了生物学上。德尔布吕克认为，经典物理学和化学不能为基因如何体现其功能提供充分的描述，某些其他的方法比旧式的机械论更能说明问题。他在美国创建了著名的噬菌体小组，以噬菌体为对象研究起基因来。他的小组作出的杰出贡献是确立了主管遗传的不是蛋白质，而是DNA（脱氧核糖核酸）。
关于生命是什么的物理学新观念
使现代物理学与生物学实现“互补”并取得卓越成就的，是薛定谔。薛定谔是量子力学的奠基人之一。他建立了描述微观粒子的运动方程——薛定谔方程，奠定了波动力学的基础，为此荣获了1933年的诺贝尔物理学奖。薛定谔与玻尔交往很深，与德尔布吕克也是好友，他充分了解德尔布吕克关于遗传方面的研究。薛定谔出于对科学统一的信念，以及对奇妙的生命现象，如遗传性状的不变性和新陈代谢等的深入思考，运用现代物理学的理论和方法剖析生命现象，提出了一系列有价值的见解。1944年，反映他主要思想的名著《生命是什么》问世。
薛定谔认为，复杂的生命现象是无法归结为物理学的普通定律的。我们不必为此感到沮丧，因为这是预料之中的事情。这不是因为生命体中有一种“新的力量”在起作用，而是因为生命体的构造同在物理实验室里实验过的任何东西都不一样。他说，要发现在生命物质中占支配地位的新定律，这些定律既非超物理学定律，也不能称为非物理学定律，而是迄今为止已确立的“物理学定律”，以及迄今还不了解的“物理学的其他定律”。
薛定谔在前人把新陈代谢解释为物质交换和能量交换的基础上，参照热力学定律，引入了“负熵”的概念。由热力学第二定律可知，孤立系统中不可逆的过程，其熵值总是趋向增加，系统趋于几率增大的无序状态，直至达到热力学平衡。而生命却是物质有秩序、有规律的行为，生命有机体作为宏观系统能保持自身的高度有序状态和不可几状态，避免很快衰退到平衡态，并不断向有组织性的方向进化。应当怎样解释生命物质的这种功能呢？薛定谔认为：“一个生命有机体要活着，唯一的办法就是从环境中不断地汲取负熵。……有机体就是依赖负熵为生的，或者更确切地说，新陈代谢中本质的东西，乃是使有机体成功地消除了当它活着时不得不产生的全部的熵。”他以高等动物为例，认为它们正是从极有秩序的作为食物的、复杂程度不同的有机物中不断吸取秩序，维持自身组织的高度有序水平的。
) x8 _1 E5 Z. c/ R7 R$ V
决定生物性状的遗传密码的设想
8 f0 Y$ M; h. Y1 O; ]- ~
薛定谔在《生命是什么》这部著作里，对于染色体如何决定生物体的遗传性状作了最初的设想。他发挥了德尔布吕克的思想。德尔布吕克曾猜想过，基因在世代相传中之所以保持其结构不变，是因为染色体（基因是它的一部分）是像非周期性晶体那样构造的。薛定谔提出了基因大分子是一种由同分异构元素连续组成的非周期晶体，像稳固的晶体结构一样，它的稳定是由于原子间的海特勒—伦敦键（一种化学上的共价键）的作用。他指出，染色体是以遗传密码的形式来决定生物体的遗传性状以及生物体未来发育的模式的。可是在微小的遗传物质中，怎么能包含那么多性状的密码呢？他说，就是这种同分异构的非周期晶体结构，“提供了各种可能的（异构的）排列，在它的一个很小的空间范围内，足以体现出一个复杂的‘决定’系统。真的，在这种结构里，不必有大量的原子就可以产生出几乎是无限的可能排列”。他天才地预示，莫尔斯电报密码只用点与划两种符号，如果每一组合用的符号不超过4个，就可编成30种不同的代号；如果在点与划之外再加上第3种符号，每一组合用的符号不超过10个，就可以编出88572个不同的字母。因此“就基因分子的图式来说，微型密码丝毫不错地对应于一个高度复杂的特定的发育计划，并包含了使密码发生作用的手段，这一点已经是不再难以想象的了”。
薛定谔第一次把量子力学中的“跃迁”概念用来解释基因突变的原因。他说：“突变实际上是由于基因分子中的量子跃迁所引起的，这种变化在于原子的重新排列并导致了一种同分异构的分子。”对比原子的平均热能，这种构型变化的阈能很高，以致变化的几率极低，这种罕见的变化就是自发突变，它们成为自然选择的基础。
' y( l/ Z: J7 R; T7 u4 p
现代生物学革命的契机
" A8 A1 n" A- Z# \. U- V5 j9 d
薛定谔在《生命是什么》一书里，倡导从分子水平来探索遗传机制和生命本质，引入了“负熵”、“遗传密码”等一系列新概念。在方法论上，他强调以物理学和化学的理论、方法和实验手段来研究生物学，并率先在这方面做了大胆的尝试。《生命是什么》在西方科学界产生了深刻的影响。
第二次世界大战结束期间，许多物理学家面临着职业选择。原子战争的恐怖和物理学可能导致的毁灭作用，使许多物理学家，尤其是年轻的物理学家重新考虑他们的工作能否给人类带来幸福。一些人感到量子理论的大发展时期已经过去，他们所能做的工作或许仅仅是完善理论的内容或改正某些细节而已。薛定谔在此时提出用热力学和量子力学来研究生命的本质，并认为新的物理学定律将在这种研究中被揭示出来，这对一批年轻的物理学家有着极大的诱惑力，吸引他们投身于这个充满希望的领域。1962年诺贝尔生理学奖的获得者沃森、克里克和威尔金斯，都是受薛定谔思想的影响从物理学领域转到分子生物学研究上来的年轻学者。战后，正是受《生命是什么》的影响，克里克放弃了研究基本粒子的计划，选择了“原来根本不打算涉猎的生物学”，而部分由于原子弹对物理学失去兴趣的威尔金斯，“为控制生命的高度复杂的分子结构所打动”，“第一次对生物学问题发生了浓厚的兴趣”。沃森则是在大学期间读了《生命是什么》而“深为发现基因的奥秘所吸引”。他们三人因在1953年发现DNA的双螺旋结构而荣获诺贝尔奖。
威尔金斯认为尽管薛定谔不是一个纯粹的生物学家和化学家，但他的著作之所以有影响，其中一个原因就恰恰是因为“他是作为一个物理学家来写作的，如果他作为一个正式的大分子化学家来写，或许就不会有同样的功效”。正是从一个理论物理学家的角度，薛定谔对生命物质和遗传机制的研究开辟了一种全新的途径，促成了现代生物学从定性描述到定量研究，从强调整体到重视具体机制，从强调生命与非生命的差别转到强调两者之间的统一性，从单科研究转到多学科综合研究的重大转折；同时促进了分子生物学的诞生。日本遗传学家近藤原平评价说：“给予生物学界以革命契机的是一本叫做《生命是什么》的小册子。它所起的作用正像《黑奴吁天录》这本书成为奴隶解放的南北战争的契机一样。”
Measuring Information: Shannon versus Popper
Philosophers have a notion of the epistemic "strength" or "boldness" of a proposition, or rather its information content, and perhaps have an idea from Popper or Wittgenstein that it can be measured using probability. This short note explains the advantage of the Shannon information measure used in information science, in terms of logical consistency and with a minimum of formalism.
The issue of how to quantify information has come up frequently in the literature on inductive logic (e.g. Hempel & Oppenheim (1948), Carnap and Bar-Hillel (1952)). What is agreed is that information content is a quantity attaching to propositions. When you receive the message "Supper's ready", we say that strictly the information content attaches not to that utterance but to the proposition that you have received that utterance. As such, information content can be represented as a mathematical function over sentences of a logical language, much like probability or utility functions. The common theme between different proposed measures is the principle, found in Popper and in Wittgenstein, that a proposition is informative according to how many possible situations it excludes. Popper and others have insisted that the information content of H is measured by 1-P(H) where P(H) is the logical probability of H . This means that the information content is just the ratio of possibilities excluded by H to all logical possibilities. This measure meets a basic requirement of a measure of information: namely that if B is a proposition which has a non-negligible probability given A, then A&B is more informative than A, because it is true in fewer situations. AvB, on the other hand, has less content.
However, the question of how to measure information has been decisively solved by Shannon (Shannon and Weaver (1949)) in a paper that is crucial to what is now called information technology. To show what is at stake, I will explain how Shannon derived his measure and then show why Popper's measure is unacceptable.
Shannon based his measure of information on requirements of logical consistency. Indeed his work is very similar to the Cox proof of Bayesian probability. Like Cox, Shannon set out consistency requirements on a type of formal system as mathematical constraints on a function, then showed that the functions satisfying these constraints differ only trivially from each other, and hence that there is really only one consistent measure.
To illustrate what is meant by a consistency constraint in this context, imagine that you receive two successive messages through the same channel each consisting of one letter of the alphabet. Imagine separately that you receive a single message consisting of two letters of the alphabet. It should be clear that these are different descriptions of the same situation, hence any truly propositional measure should give them the same value. Put another way, measures of information content should give the same value to "You receive 'A' followed by 'B'" as to "You receive 'AB'."
At the moment, we are concerned with measuring the information content of the message 'AB', not in the sense of how much is tells us about a particular issue, but in the sense of how much information would be required to unambiguously transmit the message down a hypothetical communication channel. This intrinsic complexity or information content is referred to in the theory as its self-information, whereas the extent to which a message is informative about whether or not H is called its cross-information on H.
With Popper, let us take 1-P(H) to measure information content, where each letter is taken as equally probable. In the first situation, the information content of the first message, whichever letter it turns out to be, is 25/26. Since there are two individual messages, the total information received is 50/26. In the second situation, the total number of possible messages (two-letter sequences) is 676. Whatever message you receive will logically exclude 675 of these messages, so the total information received is 675/676. Thus we have reached two entirely different values depending on how a particular message was described, and this serves to illustrate the problem with using a non-Shannonian measure.
Shannon's measure itself uses logarithms. The information content of a particular message A, called its surprisal, is -logP(A). It does not matter which base we use for the logarithm so long as we are consistent: this is the sense in which there are different mathematically allowable measures, but they differ so trivially that we can consider them to be one measure. When base two is used, the resulting unit of information is called a 'bit' (short for "binary digit"), a bit being the maximum amount of information that can be conveyed by the answer to a yes/no question.
In the above example, each one-letter message has a surprisal of -log21/26 = 4.7 bits, and a two-letter message has a surprisal of -log21/676 = 9.4 bits. Hence we see that the additivity requirement (that the content of two one-letter messages is that same as that of the one two-letter message) is satisfied.
Like probability and utility, information content is a propositional measure which obeys the expectation principle. If we do not know what a particular message is, but that it is the answer to a question whose possible answers are A1, A2, A3,..., An then the information content is the expectation of the information content over all possible messages, in other words the sum of -P(Ai)logP(Ai).
An information source or communications channel can be thought of as a question with one of a (possibly very large) set of possible answers.
This defines a crucial term in information theory: entropy. Calculating the expected information content for the set of possible answers to an inquiry gives us the entropy for that inquiry, which can informally be regarded as a measure of uncertainty attached to it. If a subject is irrevocably certain about an issue, in that one answer is given probability one while all others have probability zero, then the entropy is zero. When we have a finite set of mutually exclusive hypotheses with no information to discriminate between them, then entropy is at its maximum when all are given the same probability./p>
Since information content measures are simply descriptions of probability functions, it may seem that we do not gain anything by talking of information content that can not be expressed in terms of probability. However, information theory gives us a perspective on inferential tasks that we can miss if we talk entirely in terms of probability. To illustrate this I will consider a standard example. In a particular city, it rains three days out of four. The local weatherman makes a forecast for rain half the time, no rain the rest of the time. His predictions are such that he correctly predicts rain half the time, correctly predicts no rain a quarter of the time and incorrectly predicts no rain the remaining quarter of the time. This can be expressed in the following table of probabilities.
Here is the problem. Someone who predicts rain for every day will be right 75% of the time. Someone who accepts the weatherman's forecast also has a probability of 75% of being right on any given day. So why is the weatherman any use? The answer is that we do not simply have to accept the weatherman's forecast but use it as an information source. In other words, rather than taking the "No Rain" forecast uncritically, we can conditionalise on it to get a new probability of rain (in this case 50%).
We can evaluate how informative the forecaster is about the weather by measuring the reduction in entropy: a perfectly reliable forecaster would reduce the entropy to zero. The entropy resulting from consulting the weather forecaster is zero if the forecast is for rain and one bit if the forecast is no rain. Since these are equally likely, the overall entropy is half a bit. If we do not consult the weatherman, then given just the 75% chance of rain on any one day, the entropy is .811 . So the benefit of this forecaster is a .311 bit reduction in entropy.
By measuring the information content of the predictions in this way, we have a basis for comparison of weather forecasters (or other predictors) which is more meaningful than merely taking the probability of them being correct.
Measuring Information: Shannon versus Popper
Measuring Information: Shannon versus Popper
Abstract
Philosophers have a notion of the epistemic "strength" or "boldness" of a proposition, or rather its information content, and perhaps have an idea from Popper or Wittgenstein that it can be measured using probability. This short note explains the advantage of the Shannon information measure used in information science, in terms of logical consistency and with a minimum of formalism.
The issue of how to quantify information has come up frequently in the literature on inductive logic (e.g. Hempel & Oppenheim (1948), Carnap and Bar-Hillel (1952)). What is agreed is that information content is a quantity attaching to propositions. When you receive the message "Supper's ready", we say that strictly the information content attaches not to that utterance but to the proposition that you have received that utterance. As such, information content can be represented as a mathematical function over sentences of a logical language, much like probability or utility functions. The common theme between different proposed measures is the principle, found in Popper and in Wittgenstein, that a proposition is informative according to how many possible situations it excludes. Popper and others have insisted that the information content of H is measured by 1-P(H) where P(H) is the logical probability of H . This means that the information content is just the ratio of possibilities excluded by H to all logical possibilities. This measure meets a basic requirement of a measure of information: namely that if B is a proposition which has a non-negligible probability given A, then A&B is more informative than A, because it is true in fewer situations. AvB, on the other hand, has less content.
However, the question of how to measure information has been decisively solved by Shannon (Shannon and Weaver (1949)) in a paper that is crucial to what is now called information technology. To show what is at stake, I will explain how Shannon derived his measure and then show why Popper's measure is unacceptable.
Shannon based his measure of information on requirements of logical consistency. Indeed his work is very similar to the Cox proof of Bayesian probability. Like Cox, Shannon set out consistency requirements on a type of formal system as mathematical constraints on a function, then showed that the functions satisfying these constraints differ only trivially from each other, and hence that there is really only one consistent measure.
To illustrate what is meant by a consistency constraint in this context, imagine that you receive two successive messages through the same channel each consisting of one letter of the alphabet. Imagine separately that you receive a single message consisting of two letters of the alphabet. It should be clear that these are different descriptions of the same situation, hence any truly propositional measure should give them the same value. Put another way, measures of information content should give the same value to "You receive 'A' followed by 'B'" as to "You receive 'AB'."
At the moment, we are concerned with measuring the information content of the message 'AB', not in the sense of how much is tells us about a particular issue, but in the sense of how much information would be required to unambiguously transmit the message down a hypothetical communication channel. This intrinsic complexity or information content is referred to in the theory as its self-information, whereas the extent to which a message is informative about whether or not H is called its cross-information on H.
With Popper, let us take 1-P(H) to measure information content, where each letter is taken as equally probable. In the first situation, the information content of the first message, whichever letter it turns out to be, is 25/26. Since there are two individual messages, the total information received is 50/26. In the second situation, the total number of possible messages (two-letter sequences) is 676. Whatever message you receive will logically exclude 675 of these messages, so the total information received is 675/676. Thus we have reached two entirely different values depending on how a particular message was described, and this serves to illustrate the problem with using a non-Shannonian measure.
Shannon's measure itself uses logarithms. The information content of a particular message A, called its surprisal, is -logP(A). It does not matter which base we use for the logarithm so long as we are consistent: this is the sense in which there are different mathematically allowable measures, but they differ so trivially that we can consider them to be one measure. When base two is used, the resulting unit of information is called a 'bit' (short for "binary digit"), a bit being the maximum amount of information that can be conveyed by the answer to a yes/no question.
In the above example, each one-letter message has a surprisal of -log21/26 = 4.7 bits, and a two-letter message has a surprisal of -log21/676 = 9.4 bits. Hence we see that the additivity requirement (that the content of two one-letter messages is that same as that of the one two-letter message) is satisfied.
Like probability and utility, information content is a propositional measure which obeys the expectation principle. If we do not know what a particular message is, but that it is the answer to a question whose possible answers are A1, A2, A3,..., An then the information content is the expectation of the information content over all possible messages, in other words the sum of -P(Ai)logP(Ai).
An information source or communications channel can be thought of as a question with one of a (possibly very large) set of possible answers.
This defines a crucial term in information theory: entropy. Calculating the expected information content for the set of possible answers to an inquiry gives us the entropy for that inquiry, which can informally be regarded as a measure of uncertainty attached to it. If a subject is irrevocably certain about an issue, in that one answer is given probability one while all others have probability zero, then the entropy is zero. When we have a finite set of mutually exclusive hypotheses with no information to discriminate between them, then entropy is at its maximum when all are given the same probability./p>
Information versus Probability
Since information content measures are simply descriptions of probability functions, it may seem that we do not gain anything by talking of information content that can not be expressed in terms of probability. However, information theory gives us a perspective on inferential tasks that we can miss if we talk entirely in terms of probability. To illustrate this I will consider a standard example. In a particular city, it rains three days out of four. The local weatherman makes a forecast for rain half the time, no rain the rest of the time. His predictions are such that he correctly predicts rain half the time, correctly predicts no rain a quarter of the time and incorrectly predicts no rain the remaining quarter of the time. This can be expressed in the following table of probabilities.
| Rain | No Rain | |
|---|---|---|
| Rain forecast | 50% | 0% |
| No Rain forecast | 25% | 25% |
We can evaluate how informative the forecaster is about the weather by measuring the reduction in entropy: a perfectly reliable forecaster would reduce the entropy to zero. The entropy resulting from consulting the weather forecaster is zero if the forecast is for rain and one bit if the forecast is no rain. Since these are equally likely, the overall entropy is half a bit. If we do not consult the weatherman, then given just the 75% chance of rain on any one day, the entropy is .811 . So the benefit of this forecaster is a .311 bit reduction in entropy.
By measuring the information content of the predictions in this way, we have a basis for comparison of weather forecasters (or other predictors) which is more meaningful than merely taking the probability of them being correct.
References
Carnap, R. and Y. Bar-Hillel., 1953. "An outline of a theory of semantic information." British Journal for the Philosophy of Science, 4, 147-157.
Hempel, C. G. and P, Oppenheim., 1948. "Studies in the logic of explanation." Philosophy of Science, 15: 135-175.
Shannon, C. E. and W. Weaver, 1949. The mathematical theory of communication. Urbana, Illinois: University of Illinois Press.
No comments:
Post a Comment