Sam Altman发布《The Gentle Singularity（温和的奇点）》

《The Gentle Singularity》是 Sam Altman 写的一篇文章，聊的是人工智能和数字超级智能往哪儿走、会给人类社会带来什么。眼下人类还没摸到完全的超级智能，但进展已经足够显眼——GPT-4 这类系统在很多维度上比人聪明，用得上它的人效率直接翻倍。Sam Altman 的判断是：未来几年 AI 会在更多领域搞出突破，比如 2026 年前后可能出现能产生新见解的系统，2027 年也许会有能干现实世界任务的机器人。

英文版

The Gentle Singularity

We are past the event horizon; the takeoff has started. Humanity is close to building digital superintelligence, and at least so far it’s much less weird than it seems like it should be.

Robots are not yet walking the streets, nor are most of us talking to AI all day. People still die of disease, we still can’t easily go to space, and there is a lot about the universe we don’t understand.

And yet, we have recently built systems that are smarter than people in many ways, and are able to significantly amplify the output of people using them. The least-likely part of the work is behind us; the scientific insights that got us to systems like GPT-4 and o3 were hard-won, but will take us very far.

AI will contribute to the world in many ways, but the gains to quality of life from AI driving faster scientific progress and increased productivity will be enormous; the future can be vastly better than the present. Scientific progress is the biggest driver of overall progress; it’s hugely exciting to think about how much more we could have.

In some big sense, ChatGPT is already more powerful than any human who has ever lived. Hundreds of millions of people rely on it every day and for increasingly important tasks; a small new capability can create a hugely positive impact; a small misalignment multiplied by hundreds of millions of people can cause a great deal of negative impact.

2025 has seen the arrival of agents that can do real cognitive work; writing computer code will never be the same. 2026 will likely see the arrival of systems that can figure out novel insights. 2027 may see the arrival of robots that can do tasks in the real world.

A lot more people will be able to create software, and art. But the world wants a lot more of both, and experts will probably still be much better than novices, as long as they embrace the new tools. Generally speaking, the ability for one person to get much more done in 2030 than they could in 2020 will be a striking change, and one many people will figure out how to benefit from.

In the most important ways, the 2030s may not be wildly different. People will still love their families, express their creativity, play games, and swim in lakes.

But in still-very-important-ways, the 2030s are likely going to be wildly different from any time that has come before. We do not know how far beyond human-level intelligence we can go, but we are about to find out.

In the 2030s, intelligence and energy—ideas, and the ability to make ideas happen—are going to become wildly abundant. These two have been the fundamental limiters on human progress for a long time; with abundant intelligence and energy (and good governance), we can theoretically have anything else.

Already we live with incredible digital intelligence, and after some initial shock, most of us are pretty used to it. Very quickly we go from being amazed that AI can generate a beautifully-written paragraph to wondering when it can generate a beautifully-written novel; or from being amazed that it can make live-saving medical diagnoses to wondering when it can develop the cures; or from being amazed it can create a small computer program to wondering when it can create an entire new company. This is how the singularity goes: wonders become routine, and then table stakes.

We already hear from scientists that they are two or three times more productive than they were before AI. Advanced AI is interesting for many reasons, but perhaps nothing is quite as significant as the fact that we can use it to do faster AI research. We may be able to discover new computing substrates, better algorithms, and who knows what else. If we can do a decade’s worth of research in a year, or a month, then the rate of progress will obviously be quite different.

From here on, the tools we have already built will help us find further scientific insights and aid us in creating better AI systems. Of course this isn’t the same thing as an AI system completely autonomously updating its own code, but nevertheless this is a larval version of recursive self-improvement.

There are other self-reinforcing loops at play. The economic value creation has started a flywheel of compounding infrastructure buildout to run these increasingly-powerful AI systems. And robots that can build other robots (and in some sense, datacenters that can build other datacenters) aren’t that far off.

If we have to make the first million humanoid robots the old-fashioned way, but then they can operate the entire supply chain—digging and refining minerals, driving trucks, running factories, etc.—to build more robots, which can build more chip fabrication facilities, data centers, etc, then the rate of progress will obviously be quite different.

As datacenter production gets automated, the cost of intelligence should eventually converge to near the cost of electricity. (People are often curious about how much energy a ChatGPT query uses; the average query uses about 0.34 watt-hours, about what an oven would use in a little over one second, or a high-efficiency lightbulb would use in a couple of minutes. It also uses about 0.000085 gallons of water; roughly one fifteenth of a teaspoon.)

The rate of technological progress will keep accelerating, and it will continue to be the case that people are capable of adapting to almost anything. There will be very hard parts like whole classes of jobs going away, but on the other hand the world will be getting so much richer so quickly that we’ll be able to seriously entertain new policy ideas we never could before. We probably won’t adopt a new social contract all at once, but when we look back in a few decades, the gradual changes will have amounted to something big.

If history is any guide, we will figure out new things to do and new things to want, and assimilate new tools quickly (job change after the industrial revolution is a good recent example). Expectations will go up, but capabilities will go up equally quickly, and we’ll all get better stuff. We will build ever-more-wonderful things for each other. People have a long-term important and curious advantage over AI: we are hard-wired to care about other people and what they think and do, and we don’t care very much about machines.

A subsistence farmer from a thousand years ago would look at what many of us do and say we have fake jobs, and think that we are just playing games to entertain ourselves since we have plenty of food and unimaginable luxuries. I hope we will look at the jobs a thousand years in the future and think they are very fake jobs, and I have no doubt they will feel incredibly important and satisfying to the people doing them.

The rate of new wonders being achieved will be immense. It’s hard to even imagine today what we will have discovered by 2035; maybe we will go from solving high-energy physics one year to beginning space colonization the next year; or from a major materials science breakthrough one year to true high-bandwidth brain-computer interfaces the next year. Many people will choose to live their lives in much the same way, but at least some people will probably decide to “plug in”.

Looking forward, this sounds hard to wrap our heads around. But probably living through it will feel impressive but manageable. From a relativistic perspective, the singularity happens bit by bit, and the merge happens slowly. We are climbing the long arc of exponential technological progress; it always looks vertical looking forward and flat going backwards, but it’s one smooth curve. (Think back to 2020, and what it would have sounded like to have something close to AGI by 2025, versus what the last 5 years have actually been like.)

There are serious challenges to confront along with the huge upsides. We do need to solve the safety issues, technically and societally, but then it’s critically important to widely distribute access to superintelligence given the economic implications. The best path forward might be something like:

1. Solve the alignment problem, meaning that we can robustly guarantee that we get AI systems to learn and act towards what we collectively really want over the long-term (social media feeds are an example of misaligned AI; the algorithms that power those are incredible at getting you to keep scrolling and clearly understand your short-term preferences, but they do so by exploiting something in your brain that overrides your long-term preference).
2. Then focus on making superintelligence cheap, widely available, and not too concentrated with any person, company, or country. Society is resilient, creative, and adapts quickly. If we can harness the collective will and wisdom of people, then although we’ll make plenty of mistakes and some things will go really wrong, we will learn and adapt quickly and be able to use this technology to get maximum upside and minimal downside. Giving users a lot of freedom, within broad bounds society has to decide on, seems very important. The sooner the world can start a conversation about what these broad bounds are and how we define collective alignment, the better.

We (the whole industry, not just OpenAI) are building a brain for the world. It will be extremely personalized and easy for everyone to use; we will be limited by good ideas. For a long time, technical people in the startup industry have made fun of “the idea guys”; people who had an idea and were looking for a team to build it. It now looks to me like they are about to have their day in the sun.

OpenAI is a lot of things now, but before anything else, we are a superintelligence research company. We have a lot of work in front of us, but most of the path in front of us is now lit, and the dark areas are receding fast. We feel extraordinarily grateful to get to do what we do.

Intelligence too cheap to meter is well within grasp. This may sound crazy to say, but if we told you back in 2020 we were going to be where we are today, it probably sounded more crazy than our current predictions about 2030.

May we scale smoothly, exponentially and uneventfully through superintelligence.

中文版

温和的奇点

我们已经越过事件视界，起飞开始了。人类离造出数字超级智能很近，而且至少到目前为止，这一切远没有想象中那么古怪。

机器人还没走上街头，大多数人也还没到整天跟 AI 说话的地步。人依然会死于疾病，上太空依旧不容易，宇宙里不懂的东西还堆在那儿。

但话说回来，我们最近造出的系统在很多方面已经比人聪明，还能把使用者的产出放大一大截。最难的那部分已经过去——把 GPT-4 和 o3 送到我们手里的科学突破来之不易，但它们会带我们走得很远。

AI 会在很多方面改变世界，而最大的红利，来自 AI 推着科学进步提速、生产力抬高，生活质量的涨幅会大得惊人；未来可以比现在好太多。科学进步是所有进步里最大的那台发动机——一想到我们能因此多拿到多少东西，就足够让人兴奋。

从某个大意义上讲，ChatGPT 已经比历史上任何一个人都更有力量。每天有数亿人在依赖它，而且用在越来越要紧的事情上；一点新增能力，就能砸出巨大的正面影响；一点小小的不对齐，乘以数亿用户，也能酿成很大的负面后果。

2025 年，能干真正认知活儿的人工智能体来了，写代码这件事从此回不去了。2026 年，大概率会出现能琢磨出原创见解的系统。2027 年，也许就能看到能在现实世界里干活的机器人。

能写软件、能搞艺术的人会多得多。但世界对这两样的需求也会水涨船高；只要拥抱新工具，行家大概率还是比新手强得多。总体来说，2030 年一个人能干成的事远超 2020 年，这个变化会很扎眼，也会有很多人琢磨出怎么从中受益。

在最要紧的地方，2030 年代未必天翻地覆。人照样爱家人、照样搞创作、照样玩游戏、照样在湖里游泳。

但在同样很重要的另一些方面，2030 年代大概会和此前任何时代都不一样。人类的智能上限在哪儿，我们不知道，不过马上就能见分晓。

到了 2030 年代，智能和能源——也就是想法，加上把想法变成现实的本事——会变得极其充裕。长久以来，正是这两样卡着人类进步的脖子；智能和能源管够（再配上好的治理），理论上其他一切都能有。

我们已经和强悍的数字智能朝夕相处，最初的震惊过后，大多数人也差不多习惯了。惊叹的时间越来越短：上一秒还在惊 AI 能写出一段漂亮文字，下一秒就开始催它写一本漂亮的小说；刚惊讶它能救命般的诊断，就想知道它什么时候能把药研发出来；刚惊讶它能写个小小程序，就开始盘算它什么时候能开一整家公司。奇点就是这么展开的：奇迹先是变成日常，然后变成底线。

科学家已经在说，他们比用 AI 之前高效了的两到三倍。高级 AI 有意思的地方很多，但最要命的一条，大概是我们能拿它来加速 AI 研究本身。新的计算基底、更好的算法，还有天知道什么别的东西，也许都能被挖出来。要是一年甚至一个月就能干完十年的研究，进步的速度显然就不是一个量级了。

从现在起，已经造出来的这些工具会帮我们找到更多科学洞见，帮我们造出更好的 AI 系统。这当然不等于 AI 完全自主地改自己的代码，但说白了，这就是递归自我改进的幼虫形态。

还有别的自我强化循环在转。经济价值的创造已经启动了飞轮：为了跑这些越来越强的 AI 系统，基础设施在复利式地扩建。而能干造机器人这活儿的机器人（某种意义上，还有能建数据中心的数据中心），离我们也并不遥远。

假如头一百万个人形机器人还得靠老办法造，但之后它们能操盘整条供应链——挖矿、炼矿、开卡车、办工厂——去造更多机器人，机器人再芯片厂、数据中心，那进步的速度显然又是一回事。

随着数据中心的生产自动化，智能的成本最终应该会和电费看齐。（老有人好奇 ChatGPT 一次查询耗多少电：平均约 0.34 瓦时，大致等于烤箱运转一秒多，或者高效灯泡亮几分钟。水也大约用 0.000085 加仑，约等于十五分之一茶匙。）

技术进步会一直加速，而人照样几乎什么都能适应。会有很难的关口，比如整类整类的工作消失；但另一边，世界富得这么快，我们真有底气去认真琢磨以前想都不敢想的新政策。新的社会契约多半不会一口气立起来，可几十年后回头看，这些逐步的变化攒起来就是件大事。

照历史的老路看，我们总能找到新的事做、新的东西想要，新工具也很快消化掉（工业革命后的职业变迁就是最近的好例子）。期望会抬高，能力也同样快地抬高，大家拿到的东西都会更好。我们会为彼此造出越来越多美妙的东西。在长线上，人类有个奇怪但对 AI 很重要的优势：我们天生在意别人、在意别人怎么想怎么做，对机器却没什么感情。

一千年前自给自足的农民要是看看我们现在干的活，准说这些都是假工作——吃穿不愁、奢华得想都想不到，我们不过是在玩游戏哄自己。我希望一千年后我们看那时的新工作，也会觉得那是「非常假的工作」，但我毫不怀疑，干那些活儿的人会觉得自己的事无比重要、无比充实。

新奇迹兑现的速度会非常猛。到今天，我们甚至很难想象 2035 年会发现什么；说不定今年还在解高能物理，明年就着手太空殖民；或者今年材料科学大突破，明年就迎来真正高带宽的脑机接口。很多人会照旧过自己的日子，但至少会有一些人决定「接入系统」。

往前看，这些听着就让人脑子转不过弯。但真身处其中，多半是震撼但还能应付。用相对论的眼光看，奇点是一点点来的，融合是慢慢发生的。我们正沿着技术指数进步的长弧线往上爬：往前看永远陡得像垂直，回头看永远平得像直线，但它其实是一条平滑的曲线。（想想 2020 年，如果当时说 2025 年就接近 AGI，会显得多离谱；再对比这五年实际过成了什么样。）

当然，巨大的收益旁边就摆着严峻的挑战。安全问题得在技术和社会两个层面解决；解决之后，同样关键的是把超级智能的使用权尽量铺开，因为这事关经济格局。最好的前路可能大概是这样：

先把「对齐问题」解决掉——就是我们有把握让 AI 系统长期学习并按照我们集体真正想要的方向行事（社交媒体的信息流就是不对齐的例子：推荐算法把你刷得停不下来，对你短期偏好摸得门儿清，但它是靠利用你大脑里压制长期偏好的那个开关做到的）。

然后，着力把超级智能做得便宜、普及，别攥在某个个人、某家公司或某个国家手里攥得太紧。社会是有韧性、有创造力、适应得很快的。

只要能调动起集体的意志和智慧，哪怕会犯不少错、有些事先真会砸锅，我们也能快学快改，把这项技术的收益榨到最大、风险压到最小。在社会必须划定的大边界内给用户足够的自由，这一点非常要紧。全世界越早开始讨论这些边界是什么、「集体对齐」怎么定义，越好。

我们（整个行业，不只是 OpenAI）正在为这个世界造一颗「大脑」。

它会高度个性化、人人都用得上；瓶颈反而是我们有足够多的好点子。很长一段时间里，创业圈的技术人总爱笑话那些「想法先生」——手里有个点子，满世界找团队实现。现在看，他们的出头之日怕是要到了。

如今的 OpenAI 身兼很多角色，但根子上，我们是一家超级智能研究公司。前面的活还多得很，但脚下大部分路已经被照亮，黑暗的区域正在快速退却。能做这件事，我们心怀感激。

「智能便宜到无需计量」已经在伸手可及的范围里。这话听着疯，可要是 2020 年我们告诉你今天能走到这一步，大概比我们眼下对 2030 年的预测还疯。

愿我们在通往超级智能的路上，平稳、指数级、波澜不惊地扩展。

原文链接：the-gentle-singularity