PRML Notes - 3.2 Bais-Variance Decomposition
在第一章中，我们把回归问题分解为两个部分：推演和决策。假设推演步骤已经完成，即后验分布\( p(t|\mathbf{x})\)已经得到，接下来就需要选择一种损失函数，并以最小化损失为目的进行决策的步骤。一种常用的损失函数为平方损失（注： 请明确区分推演过程中使用的平方误差与决策步骤中使用的平方损失），在此条件下，最优预测值是通过以下条件期望得到的
\[ h(\mathbf{x}) = E[t|\mathbf{x}] = \int t p(t|\mathbf{x}) dt \]
平方损失可以写为以下形式
\[ E[L] = \int (y(x) - h(x))^2 p(x) dx + \int (h(x) - t)^2 p(x, t) dx dt \]
其中，第一项表达我们得到的预测与最优预测之间的差异，而第二项则表达了最优预测与真实响应值之间的差异，这项实际是以噪声引起的，是我们是不可控的。理想情况下，我们的预测与最优预测一致，这样就完全去掉了第一项，从而最小化平方损失。
实 际中，这个最优预测\( y(\mathbf{x}) = h(\mathbf{x}) \)是无法得到的，因为它的计算需要无限多的数据，而我们只有有限个数据点。因此，我们只能寄望于先找到一个模型\( y(\mathbf{x}, \mathbf{w}) \)，再通过训练得到最优参数以最大程度的接近于\( h(\mathbf{x}) \)。
由 于数据是有限的，因此通过这些数据来训练而得到的模型是具有不确定性的。从贝叶斯的角度来看，这种不确定性是通过参数\( \mathbf{w} \)的分布来表达的。而在频率框架下，最终得到的并不是一个关于参数的分布，而是一个关于参数的点估计。不同的数据样本对应不同的参数点估计值，其不确定 性表达为不同数据训练造成的参数的波动以及损失值的波动，亦即
\[ E_D[(y(\mathbf{x}; D) - h(\mathbf{x}))^2] = (E_D[y(\mathbf{x}; D) - h(\mathbf{x})])^2 + E_D[(y(\mathbf{x}; D) - E_D[y(\mathbf{x}; D)])^2] \]
其中，第一项表达了该模型的平均预测与最优预测的差异（bias），而第二项则表达了该模型预测值自身的方差（variance）。结合前两个表达式，我们可以得到以下等式关系
\[ expected\_loss = (bias)^2 + variance + noise \]
我 们希望最小化损失值，但是实际上我们将不得在bias和variance之间做出权衡。当我们选用了一个非常复杂灵活的模型时，它能够很有效地降低 bias，却引入了较大的vairance；相反地，如果我们选用了一个严格的模型，它的variance较低，但是却引入了bias风险。最优的模型选 择就是找到bias和variance之间的一个最优点。
这个Bias-Variance Decomposition在实际中应用价值并不大，因为它需要计算\( E_D \)，而这是不可能的。理论意义上，这个分解式还是给我们带来了一些insightful ideas.
2022年8月20日 15:34
Tamilnadu Board Model Paper 2023 Class 5 Pdf Download with Answers for Tamil Medium, English Medium, Hindi Medium, Urdu Medium & Students for Small Answers, Long Answer, Very Long Answer Questions, and Essay Type Questions to Term1 & Term2 Exams at official website. TN Board Question Paper Class 5 New Exam Scheme or Question Pattern for Sammittive Assignment Exams (SA1 & SA2): Very Long Answer (VLA), Long Answer (LA), Small Answer (SA), Very Small Answer (VSA), Single Answer, Multiple Choice and etc.
2023年2月04日 00:58
Mahanagar Telecom Nigam Limited does give a variety of options to its customers which can be availed online for MTNL Mumbai bill payment and for a plan change through the dedicated portal which customers of the zone can easily track their bills and service records. MTNL Duplicate Bill It makes it easy for customers to track their dues and as well select a better plan in future days, and the switching of plans is far east through the Online MTNL Mumbai web portal.
2023年7月02日 10:13
Our reporting team plans to release the Education & Recruitment Update for all age groups and provide inside coverage to show the real picture of current occurrences. As we want to provide news divided into General, Political, Crime, Sports, Entertainment, Education, and World News, our goal is to meet the needs sample-paper.in of people of all age groups.Professional writers have gathered for specialised news coverage of the most recent events in the nation (India). Our team is made up of professional writers and citizen journalists with a wide range of journalism interests who are committed about delivering education updates in the public interest while maintaining transparency.
2023年8月23日 14:58
All the basic information about every celebrity is available now on the largest database of celeb height wiki where you can find all the information and net worth of a singer, actor, businessman...
2023年8月26日 19:54
Arunachal Pradesh 8th Question Paper 2024 All the Online PDF format Question Papers Provided by Board of Secondary Education Arunachal Pradesh,Arunachal Pradesh Model Question Paper 2024 are Arunachal Pradesh 8th Question Paper 2024 Prepared as per the updated Board of Education in the Arunachal Pradesh State Syllabus to help Students in their exam Preparations, which Aims to build both the Practical knowledge as well as analytical Skills for Arunachal Pradesh Every Student,Arunachal Pradesh Question Paper 2024 are Prepared as per the updated Arunachal Pradesh Syllabus & Exam Pattern to help Students in their Exam Preparations.Arunachal Pradesh Previous Question Paper 2024 will help a lot to the Students.