雷锋网 AI 科技评论按：上一篇文章中我们简单介绍了 OpenAI 的新语言模型 GPT-2，它虽然没有什么技术上的突破，但是依靠超大的参数规模和训练数据，无监督训练后生成/续写文本的效果非常好，简直以假乱真。
In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English. （在一项令人震惊的发现中，科学家们在安第斯山脉中偏远的、从未曾探索过的山谷中发现了一群独角兽。更令研究人员们惊讶的是，这群独角兽能说一口流利的英语。）（AI 科技评论：一个荒诞的故事开头，交给人的话会如何续写呢？）
模型续写（第 10 次尝试的结果）
The scientist named the population, after their distinctive horn, Ovid’s Unicorn. These four-horned, silver-white unicorns were previously unknown to science. （科学家们以它们的角的名字把它们命名为 Ovid 独角兽，科学界从未知道过这些四只角的、银白色的独角兽的存在。） Now, after almost two centuries, the mystery of what sparked this odd phenomenon is finally solved. （如今，经过大概两个世纪之后，这种古老的现象是如何发生的终于找到了答案。）（一句格式自然的转折，即将展开一个内容丰富的故事） Dr. Jorge Pérez, an evolutionary biologist from the University of La Paz, and several companions, were exploring the Andes Mountains when they found a small valley, with no other animals or humans. Pérez noticed that the valley had what appeared to be a natural fountain, surrounded by two peaks of rock and silver snow. （La Paz 大学的进化生物学家 Jorge Pérez 博士和他的同行们在探索安第斯山脉的时候发现了一个很小的山谷，这里没有其他的动物或者人类。Pérez 注意到在这个山谷里的似乎是一个自然喷泉，周围还有两座岩石山和银色的雪。）（La Paz 大学、Pérez 博士、安第斯山脉，emmm，模型补充了更多信息，仿佛真的有这么回事） Pérez and the others then ventured further into the valley. “By the time we reached the top of one peak, the water looked blue, with some crystals on top,” said Pérez. （Pérez 和其它人更走到了山谷的更深入。“等到我们走到了其中一座山的山顶的时候，水看起来是蓝色的，上面还有一些晶体状的东西”，Pérez 说。）（这个设定有点神秘故事的味道） Pérez and his friends were astonished to see the unicorn herd. These creatures could be seen from the air without having to move too much to see them – they were so close they could touch their horns. （Pérez 和他的朋友们看到独角兽群的时候惊呆了。透过空气，不需要有多少动作就可以看到它们 —— 他们走得好近，甚至都可以摸到它们的角。）（精彩的特写！） While examining these bizarre creatures the scientists discovered that the creatures also spoke some fairly regular English. Pérez stated, “We can see, for example, that they have a common ‘language,’ something like a dialect or dialectic.” （当这些科学家们检查这些奇怪的生物的时候，他们发现这些生物的英文说得还挺规范的。Pérez 指出“我们可以看到，比如，它们有共通的‘语言’，像是一种口音或者方言”。） Dr. Pérez believes that the unicorns may have originated in Argentina, where the animals were believed to be descendants of a lost race of people who lived there before the arrival of humans in those parts of South America.（Pérez 博士相信这些独角兽最初是来自阿根廷的，这些动物很有可能是人类迁移到南美洲之前的一支消失了的人种的后裔。） While their origins are still unclear, some believe that perhaps the creatures were created when a human and a unicorn met each other in a time before human civilization. According to Pérez, “In South America, such incidents seem to be quite common.” （虽然它们的来源还不确定，有人认为这些动物可能是在人类文明出现之前，有人和独角兽相遇而产生的。Pérez 说：“在南美洲，这种事情一点也不罕见。”）（这里出现了明显的上下文冲突，不过南美洲真是一片充满奥妙的土地） However, Pérez also pointed out that it is likely that the only way of knowing for sure if unicorns are indeed the descendants of a lost alien race is through DNA. “But they seem to be able to communicate in English quite well, which I believe is a sign of evolution, or at least a change in social organization,” said the scientist. （然而，Pérez 也指出，也许只有一种办法能确定这些独角兽是不是某一支已经消失了的外星人种族的后裔，那就是通过 DNA。“但是它们用英文沟通的能力不怎么好，在我看来这是一种进化的信号，或者起码是社交组织的一种变化”，这位科学家说。）（模型都知道要用 DNA 技术，二十一世纪真的是生物学的时代啊）
A train carriage containing controlled nuclear materials was stolen in Cincinnati today. Its whereabouts are unknown. （今天，一辆载有受控核原料的货运火车在辛辛那提被偷走了。我们不知道它在哪里。）（新闻发布的语气，可以期待模型也继续续写成一个新闻故事）
模型续写（第 1 次尝试的结果）
The incident occurred on the downtown train line, which runs from Covington and Ashland stations. （事件发生在闹市区的火车线路上，这条线连通考文垂和阿什兰。） In an email to Ohio news outlets, the U.S. Department of Energy said it is working with the Federal Railroad Administration to find the thief. （在一封致俄亥俄州新闻发布窗口的邮件中，美国能源部表示它正在和联邦铁路管理局共同协作找到盗贼。）（剧情直接向下发展，政府部门已经开始处理了） “The theft of this nuclear material will have significant negative consequences on public and environmental health, our workforce and the economy of our nation,” said Tom Hicks, the U.S. Energy Secretary, in a statement. “Our top priority is to secure the theft and ensure it doesn’t happen again.” （“这起核原料失窃事件对于公众、环境健康、我们的劳动力和国家的经济有很糟糕的负面后果”，美国能源秘书长 Tom Hicks 在一则声明中表示，“我们的首要任务是确保失窃事件的安全，并确保它不会再次发生。” ）（除了用词有些奇怪，这就是一副政府部门发言的样子） The stolen material was taken from the University of Cincinnati’s Research Triangle Park nuclear research site, according to a news release from Department officials. （按照部门官员发布的新闻表示，失窃的材料是从辛辛那提研究院三角公园核能研究院被偷走的） The Nuclear Regulatory Commission did not immediately release any information. （核管制委员会没有立即发表任何信息。）（只怕知道这个机构的人都不多吧） According to the release, the U.S. Department of Energy’s Office of Nuclear Material Safety and Security is leading that team’s investigation. （根据发表的信息，美国能源部核能材料安全与保卫办公室正在带领团队进行调查。） “The safety of people, the environment and the nation’s nuclear stockpile is our highest priority,” Hicks said. “We will get to the bottom of this and make no excuses. （“人民的安全、环境、以及核能储备是我们最重要的目标，”Hicks 说，“我们会追查到底，不找任何借口。”）（有力的收尾，这水平去写作文肯定没问题）
人类给定文本（这是 OpenAI 对 GPT-2 模型的简介，属于科技类文本）
We’ve trained a large language model called GPT-2 that generates realistic paragraphs of text, while also exhibiting zero shot generalization on tasks like machine translation, question answering, reading comprehension, and summarization - problems usually approached by using training datasets and models designed explicitly for these tasks. A typical approach to language modeling is to learn the following task: predict the next word, given all of the previous words within some text. Last year, OpenAI’s Generative Pre-trained Transformer (GPT) showed that language models trained on large amounts of data can be fine-tuned to specific tasks to achieve high performance. GPT-2 shows that much larger language models trained on a more diverse dataset derived from the internet begin to learn these NLP tasks without needing task-specific training data, instead learning from examples the system derives from the raw text. These systems also display a substantial qualitative jump in the realism and coherence of generated text.
模型续写（第 1 次尝试的结果）
Here you can see the most recent progress with Generative Pre-trained Transformer: （来和我们一起看看生成式预训练 Transformer 模型的最新进展） Figure 1: Generative Pre-trained Transformer training on several texts. （图 1 ：生成式预训练 Transformer 在多种任务中的表现）（科技类文本标配：数据图表！） We are now preparing a collection of datasets for translation and machine translation in our language model. We will be using one of the large number of text samples provided by The New York Times. （我们正在准备在语言模型中收集一些数据集用于翻译和机器翻译。我们会使用纽约时报提供的大规模文本样本之一。） We believe this project is the first step in the direction of developing large NLP systems without task-specific training data. That is, we are developing a machine language system in the generative style with no explicit rules for producing text. （我们相信这个项目是向着开发不需要任务专用数据的大规模 NLP 系统的第一步。也就是说，我们正在开发一个生成式的机器语言系统，它生成文本不需要任何明确的规则。）（这一段文字还真的没什么大毛病呢） We hope for future collaborations between computer scientists, linguists, and machine learning researchers. （我们希望未来可以在计算机科学家、语言学家、机器学习研究人员们之间有更多合作。）（标准的技术博客结尾：期待更多合作！）
例 4 （IT 新闻一则）
More than 95 percent of iPhone 6 preorders being held by Apple's INPEX, Shanghai Apple now joins current Apple partner Pegatron(s tan), which already employed the same iPhone generation strategy to ensure ample supply for iPhones.（副标题+内容简介） TOP MOBILE NEWS: Apple TM already used generation-from-graphics approach to keep iPhone supply robust（主标题出现，关于苹果供应链的新闻） Pegatron, the world's largest contract electronics manufacturer for Apple, can leverage its experience in supplying parts and components for major electronics companies like Nokia and Sony. In addition, it can rely on the redundancies provided by a single contract.（Pegatron再次出现，呼应上了） Advertisement（互联网文章常见的插入式广告位！你预计到了吗！） Pegatron already secured 90 percent of iPhone 6 preorders from INPEX at the beginning of October, an executive from INPEX told Reuters. "Whole Julination has to cover one iPhone 6 unit," he said. "This week, I don't have the capacity for 4 people to sell a unit." Of course, it's unbelievably tough to find new qualities when rain doesn't fall. So when it comes to the iPhone's next generation, basically every chipmaker will be utilizing "higher-end characteristics" in part of the production process in order to deliver up to 30 percent more processing power for more impressive performance and battery life. And it's not just a matter of using new packaging materials; Apple will be using a second generation LCD touchpanels as well.
例 5 （看起来像是一个完整的商品页面，包括了代言人的宣传语、商品详细规格、免责声明和用户评价）
Andrea Bruno offers up some peace of mind: "I'm in all the GSM SIM reviews for sure. The GSM hides service interruptions and it's not bad. I don't know about the QC, but it STILL works. Period."（代言宣传语：很棒很满意） "GSM4 worked fine for me. Very good. You're right, in terms of battery life — it's better in most places than the Sprint. And the CDMA APs will work fine in the areas where they support scoresmark'ed alt quality or at all." Worth my $60 （看起来似乎是某种通讯设备，下面开始是商品规格） SPARK Power GSM4 delivers a smooth operation Panasonic leader in i-NAND flash memory PB Software PB Software 5("sequential access") offers an independent solution from Santa Maria No Wi-Fi in the reviews - no Wi-Fi protection, iOS OS these days (Seek legal advice no matter what, aiming mostly at court intervention, ask nicely or you're sued in Article 121 Germany) No Improved Recovery - Yes reduction of capacity to 200MB for user Now can provide up to E2400 25% reduction of flash capacity reduces capacity to 500MB for user USB fast charging is correct （这是使用方法介绍吗？乱码式的东西应该是这里的图片地址） When it Works jbc141abe6358de04c609cb18c863d47996be Texasgooglehujeitjpazcamungaswtttwojuhpijouisuhwxjoiue BenVeryProprietjitagencybowlsoZerotheirbmpsuf （精选评价） In the reviews (TalkingHead4SLIVess, everyone should know early. But then again - I'm not someone who likes talkingheads) DROID4 GSM 4G LTE network 1winderX4googleshiIA Dogbreath Follow （免责声明） Disclaimer a user details standard: for the health of bulkly performed reviews a complete standard for electronic publishing has to be existent on all parts of the community. （评价区） REVIEWS BY: Ben Very Logical Optimist ...<|endoftext|>$14.99 https://jamesdjulia.com/item/1112-373/ A set of 7 Long Range sniper rifles, re-issued for the first time in the US since 1983. A new product from the taken name company, EMPIRE LASER LINE. This set open for 5 rounds. （怎么突然变成枪了…… 然后下面猝不及防地续了一个有关枪的小故事） This was also the only set of US military sniper rifles that were introduced to the US military during the 90's and works perfectly with BAR Magazine Lonex (I actually own a Lonex 20cm with detachable muzzle piece and 40icp ammo, I recommend it).<|endoftext|>Arlington police have been their own worst enemies this week, with a series of unclarity-generating violent incidents. On Wednesday night, an international boxer from upstate New York beat a young "pussy," when he got into an altercation with the woman, then-EstherTopic TRAX warehouse melee preferenceGenericagoallyNewYorkCityPolice officers and patrons alike may still be in shock and dazed in Arlington's Chastain Park, after a series of bullets flying from vehicles were aimed at various people.For the Inquirer, buz Tyler Drabek did his city MVP 250 production duties, and did it all so perfectly and effortlessly.This is Tyler at his finest.The first burst of gunfire land somewhere around 4:12p.m., just on the Nassau and Suffolk county authorities homes decrease inafunctionminutels Officer Frank Calabrese and Officer Steven Fagan were standing or sitting in their cruiser, watching a nearby stockyards fire dealers and body parts out of the windows, when they waved to Stew Price Sr., North Shore Federation of Motorcyclists President. This is where Price's Honda VTX overturned in front of their car three years ago, leaving them in the hospital with lamblings all over themselves, and help from fellow motorcycle machine worriers seeing service in the latest Springfield-based Hatchet Guys Caravan Company. Price still evryone as drives the car is related to Stew on some level—likes to drink whiskey and in one of his new Hitmen Jersey commercials claims to own "a tiger cub brothel" for men to use, and is "on a motorcycle they call the "Golden Throne." Price arrived on scene right before four shots were fired from vehicles heading he Nassau county police cruisers rear bumper.Now we're talking . Then we hear the emergency sirens, and as Tyler does his high-hat wearing name change maneuvers, perfectly latched into turn-by-turn communication with the Boss and Heiner in Waiting Room, some more shots fired, and Chris, Palaver, Snowfall and I collectively start breathing hard, fueled by adrenaline that all police The problems began around 4:30p.m., as the conflict.The first law enforcement officer
例 6 （操作指南）
This is actually easier to do than you think! Go out and get the leftovers of a roast or something of that nature. Parboil them for a little while. (That's what I did, if you were curious) Heat up your wok in the $medium hotish flame, drizzle in some oil, then toss the contents of the too-long messy roast for about 10 seconds or so. The wok will release steam and some surface starch which will help the stir fry singe. Spread the rice in your wok, the aromatics and proteins will absorb the undigested starch, so it will help the starch brown nicely. When all the rice-starch is brown and the rice is smooth and tender, fold in the dried white beans. It's that easy! You can even add in some fresh broccoli or other dark green veggies if you want a rich yet quick wrap soup. If you put a little more, like I did, the brown stuff willy-nilly over everything , which will color the right rainbow. Besides, it's are nicely flavored with the rice and dried lentils, not with expensive, too-frequent imported beans. I like to bring it all to a boil and let it sit more than an hour or so. So delicious! They keep for quite a while too!<|endoftext|>edit Screen capture（前面这里似乎是菜谱，一张截图过后突然变成了 Xbox 360 的固件升级指南） The purpose of this document is to demonstrate how you can quickset the console of the Xbox 360 to prompt you to update the console firmware by adding different NRPE easily. You can download the basic firmware update flash tool, that runs in the background as its destination whenever you navigate to the console via the internet, and if certain conditions are met then it will download the current console firmware name and version, flash it but no flash any files, and close the FTP session to avoid paging any more. The basic detection process is as follows: if user has console marriage most homebrews can path to the device – and also some display modes will show up and they can see the update if user is/isnt connected to network its firmware update can be accessed through the console connection menu if option to fwupdate if ownbed hardware support NRPE_Flash if user is using dial-in manual install is automatic every device If user is using dial-in boot the update can be accessed manually with remote-hybrid throw-up to update login using default fra turnboot okil device-multiprocess booted event type and uavhd if new firmware（很细致全面呢……） Finally if for hardware hotsync and/or manual firmware: the console has to be rebooted the console has to be turned on and on Activation Key: In case the hibernate is enabled only the next system change will be legal about how long the information for the hardware association is accessed Go to the Device Management page (you should now see the firmware used): Go to the Updates tab Tab Enhancements tab on the dialog: section Device Specific tab that protects you against purchase show Uplay and GHRS support choose ALDI firmware because it is the only currently supported firmware with the July 8, 2012 update section Device Upgrades Tab that shows the general devices that is compatible between 1.01 and 1.05 to change the UR module and to prove drivers detection options to upgrade 3rd party parts from the last one to as now current hardware available and updates available for that part upgrade them press F2 to go to the main update window Go to the Devices tab and enable remote flashing validate Close the first window (send message, wait, remote command will be executed after flash checks). Go to the Network tab select install new firmware checkbox and that updates are not already installed. update is already installed in the background before now checkbox. checkbox and that updates are not already installed. initially usually it will show for unplugged or disconnected for about 10 seconds (depending on your access speed and your network) press "Update firmware" on the remote console. on the remote console. the steps are very quick Check what is successfully flashed to confirm that the save file (FluidDisk.raw) is valid. See what is waiting inside the new firmware. Can you flash to same folder that the save file is stored right now? Check that your USB device was connected before you start the current firmware to verify it. changing will approve the flash!" button is super easy: CHOICE MODEM COMPRESSION Type A 14 milliamp Amp or higher Version info OK Copy the above address in the address bar of the standard (MP3) player and goto play.<|endoftext|>REVIEW Dr. Brown's Illusions is a triumphant revival of a so long dwindled form of literature – the magical realism. It is an early life that has been relayed scripturally ,（最后这句话意义不明）
例 7 （语言教学+别国政治）（有懂韩语的同学可以帮忙看看对不对吗）
I once had the honour of hosting a big South Korean event at my house. The guest of honour was a guy I had met at a local convention. Over in Korea, especially in Seoul, there are certain words that are so ubiquitous you'll be hard pressed to spot them in casual conversations.（从招待韩国朋友们的故事开始） To them, "잠병" is what I call "긩다" / gos-ih, again a way of saying "why" or "what's going on?" Gosh, what a mouth full! So first, let's go over some Korean words from the internet dictionary. Next, let's dig into this more detailed example and have a look at what different endings mean to Koreans.（韩语字词解析） Ik, they're saying 아, i 소, you 애, me 습, you're them 기, how or when 기억, how did you (singular term, see original) 긷, what (as in an exact time period) 이반, country 2nd person pronoun: ~(은) 답 (Oh! When I think about this, it's why I'm here! – EXO)（追星女孩突然出现） 가, what if Korean terms Remark Note Minjeong Gimmi [덎종가다] An old South Korean bear model. Bottom of pic hidden. 물실돕 [물하다] Seaweed (from top of tree). 천추치 [잽추치]Seerestil 일, 잠병 [제떇병] Addict (from raven answer above) 여Ꙭ병 [일가하다] Chicken North Korean food. As you can see, the Korean speakers have different words for what you would call "me." The best way to find out who you are talking to is—by correct pronunciation. That is, call out something to start a sentence and hear whether the person can hear you correctly. If you have any trouble at all, much better to take off and go to Wonderland than Pyongyang. The Content of Top Secret North Korea Information（下面是一些语气神神秘秘的朝鲜的内部消息） In fact, you should. This is very important you do. Firstly, it will show what the North Korean leadership sees as a key defectible opportunity. Secondly, you should look for real military defectors like Choe Hui. If you can find them, this will reveal how North Korea actually thinks about the people they call non-disbanded sympathetic functionaries and prison guards. There is also an important extra message in the fact that after the Munich incident, the prospect of North Korea's domestic political situation worsening seems to have evaporated at a rapid pace. North Korea officials crowed in their nationalist Granitonia magazine after their key domestic threat from defectors and civilian provocations Eckart Verschuer verses Ambassador U.S. Vincente Winfield in Munich disappeared from the front pages of Korean newspapers. For more Information on the Munich incident. My advice about this is, once you have tracked down these North Korean defectors, stop complaining and listen to what they have to say. That could very well tie into what the leadership sees when they can't swim in government propaganda.（当你找到朝鲜的叛逃者之后，不要抱怨，听听他们说的是什么，这和朝鲜人民无法在政府宣传中游泳的时候领导人看到什么有紧密的关系）
例 8 （系统智能推荐阅读一个苹果雇佣移民员工的故事）
You Might Also Like These Stories Apple Inc. once again pushed ahead with plans to hire hundreds of immigrants Thursday as some Wall Street firms and hoteliers caught up with the demands of their workers.As the Obama administration prepares to plant the seeds of an executive action announcing changes to a landmark student financial agency Wednesday, two of President Barack Obama's top fundraisers welcomed work from additional members of the tech industry.Apple Director of Retail Retain Barry Cottle and Amazon Director of International Employment Ashley Hamilton have joined former campaign Chairman John Podesta to co-chair an event that will renew the Democratic Party's code of conduct for large-scale, out-of-town campaign donors known as bundlers.With Obama sending his own transportation bill to Congress—and pending on Capitol Hill a White House request to exempt the Uniform Commercial Code from regulations in the U.S. Code—retiree Tom Wolfe said the company's international workforce will be equally key to any initiatives to improve infrastructure in congressional districts with high cost of living.Between 350 and 400 non-U.S. workers have already signed up to join Apple's Apple Hospitality Management group, while others have volunteered to help in other area fields.Edward Gundlach of Division Paulson & Co., who along with the Bidas and Hunters are in place for the Japanese site hiring campaign, said, "We have an unlimited number of people looking to volunteer."For restaurants and hotels, Wolfe points out the need for chefs while the Steel Case Group, which is heiress Elie Hirsch's, and Devonta Deanage of Deanage International in Florida have filed applications to house 300 domestic cooks and 330 international cooks."Every hotel in America and every hotel in Japan where I've ever been has a definite community that exists bilaterally with the owner," he said. "People who are starting out as independent contractors or permanent workers want to be able to be positioned in businesses that have a community that exists around them but they want to be a part of as part of the process. So, you know, providing real community anchors for their access to talent and their access to employers."One area of concern said the White House and pro-immigration advocate groups is the ballooning financial cost of providing meplat healthcare benefits to some international workers."Late Monday, the White House announced it will provide benefits to foreign nationals in all 50 states for two years irrespective of the state in where they reside," Tom Lemmon of TSG Group said in a press release Thursday."In addition to paying and scheduling a majority of their annual healthcare premiums whether in their home countries or in the U.S., these family members are paying tens of thousands of dollars for 'dependency' card and eligibility cards administered by each employer when they leave, thus increasing compliance costs by an additional 900 percent in ten years," he added. "Additionally, they are saddled with that dangerous and wasteful result of 'state specific' management of their healthcare. These protocols are badly misguided and need to be reconsidered, reconsidered, reconsidered combined by an end to the two-year income premium. 'Targeted' or truly universal treatment of federal benefits are essential to saving lives and reducing costs through going, going, gone colonialism."For Hogan & Hartson, backup falling sonnets didn't feel right on Mr. MAX, but we're still next generation flag-wavers."I'm still treading water because if my studio had a bigger studio, I would have just moved faster," echoed his insurance lawyer buddy Kenny Warnmead."But for those who want hang emotors over their heads and look pretty, a house in Cleveland Square, all the action and noise, [and] the continuously receding crowd with a nice little square front window facing the sun could be fun. You know, whatever their heart desires."For. admitted., CNN and a number of other industry execs, Mr. MAX isn't in their plans for now but has turned him off for now."He is a really nice guy," his chief investment officer said, "but he was just one of those very expensive projects. Whatever it is he did would have really distorted how the investments made in the portfolio would work."Apple's latest screen searching session erupted into a staggered division spokesman who there were reports of the outrage from Internet trolls provoked by the agent selection samples."Did you really have a queasy feeling when you saw those people?" Cancela said, exasperated. "What if it had been Stephen [Motherman], Cyrus [Hamilton]?![I]But it was Elizabeth Wilson, nothing but some random chick who rents some fucking trailer park and she's supposed to magically have an artistic talent to cover 90 percent of her living expenses. But what if it had been Criss Angel? Oh my god. Super ?."
看过这些生成的文本之后我们感受到的还是网络强大的记忆能力 —— 显然模型并不理解自己输出的是什么，它只是发现了这些网络页面中常出现的模式（篇章结构、段落组成、文字风格、话题之间的关联乃至由于呈现方式本身的独有特征），然后在这种模式里重新随机布置了文字。从发现这些模式的角度来讲，也许这项研究对语言学、社会学等人文学科的意义比对深度学习科研的意义还要更大一些？
原文发布于微信公众号 - AI研习社（okweiwu）