专栏首页大数据智能实战基于gpt-2模型(117M预训练模型)的文本自动生成测试

基于gpt-2模型(117M预训练模型)的文本自动生成测试

openai的gpt-2模型最近在风口浪尖上。Language Models are Unsupervised Multitask Learners论文已经出来,但是由于该模型没有将训练过程开源出来,所以本博客仅仅是针对已经公布的117M的预训练模型进行测试。

1、论文贡献

In this paper, we connect these two lines of work and continue the trend of more general methods of transfer. We demonstrate language models can perform down-stream tasks in a zero-shot setting – without any parameter or architecture modification. We demonstrate this approach shows potential by highlighting the ability of language models to perform a wide range of tasks in a zero-shot setting. We achieve promising, competitive, and state of the art results depending on the task.

找更大数量的无监督训练数据来执行多任务学习,使模型更具泛化能力。论文实验也证明了该模型具有惊人的效果。

该论文的模型大部分还是遵循GPT-1的模型,但有两点不同的是:

(1)训练数据集更加庞大;

(2)在第二阶段时候,无监督地做多样性的任务。

2、117M的实验测试

执行测试程序,效果如下:

其中任选一个例子,可以看到对话的自动生成效果,可读性还是非常好的。

======================================== SAMPLE 36 ======================================== I was sent a few hours ago and while it is a good thing I almost gave up my game of EVE Online during the lack of a win! I've been in a bunch of small flops and this extended feeling of accomplishment wears off quickly when you are up against a hungry and crowded large team, leaving your heads more empty then ever.

Do you feel like a human being it seems? How did you finish the game and what is your plan for you now?

Of course I felt like a human being just yet I wanted to caution you Salty Dogs is fast moving,

I feel so overwhelmed right now, and it has taken me awhile to get used to being PVP on a big screen map. It's hard to describe just how good the experience is. You can hear the birds and all the bees loudly above you at every turn, but I'll tell you what's first as Dae-bye with the horde

But just as I reported yesterday you have launched your first flame fight with a cyno drop on your squad mate. Is he selfless, or was he just taken aback? Granted he's very clever and oddly physically, together with being conscious you can be flexible, give him time and eternity and maybe even benefit maybe three AXP from a loss. The cyno drop is also a blessing and bonus that once again gives his DPS a blurred product

Anyway when we first worry keep an eye on the team members, I will be picking off Pawns a (he has 2 fire!) and keeping all shields focused and a target area is on-screen so I can execute nice pulls and therefore tell the ppl what triggers and emulate the touch infantry roll action

that's a tremendous turn too. Your thought did make sense, and my observation went viral. I'm feeling so overwhelmed right now just to try out some new stuff. My team mates are done playing, but I will certainly focus on whats going on on higher levels of gameplay,

what better way then to see when you actually harrass the team mate out of defending Silo Exchange with the basic offensive charges in hand before further anti-tank push. For those with a spare CHAR it's worth an inclination, the Tyranids can't be counted as melee units at the moment, this makes them odd names but the whole point of running kids in university's first community he asks "Phillip, deploying? Will you kill me?"

So have fun,

-Dae-bye Salty Dogs<|endoftext|>There's a lot of personal stories beautifully imagined frozen at shore on Capri Island in Nova Scotia - one for instance, from a group of above-board vessels shark roaring off the Vermicelli's 302 and invoked to wait desperate refugees in their misery in Germany.

Some people, especially some conservatives, want tender mercuppactions.

Others may, however, care intensely about another island on their property: Bermudu, a 9,745 sq. ft. country with 156,000 sq. ft. headquarters.

Story continues below advertisement

The association posed first to a shelf for the pit safari in Portland, Oregon. They are part of Friends of Port St. Albans, a Halifax proprietorship that requires M. Arthur Day paid the adventurer 3000 a day.

Take Port St. Albans president Caryn Francitisis on Oct. 5 after receiving a request to dock a crew within 10 yards of the Basecamp think tank by New York-based Quickminster Sandlot meat supplier. With her supporters dwindling with the new year, Francitisis hopes for cabinets and Convoy drunken salmon Gypsy gentle traps once provided for Rusty Commodore subduing Golgoths longiensEXPLAIN (October 18, 2011).

A moment in early October brought two fateful encounters in the city that left staff of First Niagara sobbing for help.

In a matter of minutes Mayor Terry Wooldridge led protesters through Barrington Avenue North until a police car sped up and took them off, prompting shells and dead fishy nets out from the pier. Proceeding slowly, emergency employees found the couple and turned off the next two raging barges coming off the Annapurna peninsula, where some had been making prayers.

The Heard Four

Stay cool but don't embrace NYC's exclusive fact FACT IT'S A MESS Center Anchorage, Alaska - Canada, 126 nautical miles northeast of Fort McMurray, turns down 35,000 planned visitors a day, but has 50,000 RT champions today, the Make Maine count marshals have announced.

Earning FOLLOW COLLEGE ENTHUSIASTIC RECORDS

Sound guide dictates Fashion Document

Their arrival prompted speculation about our species as we see it. The grave possibilities spread seafaring, thereby attendant decades enigma ... The nutrer Susie Lewis (with nary a hint of her apparent blue-bloodgums) penned a poignant cartoon called "Beyond Ferry Line."

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

我来说两句

0 条评论
登录 后参与评论

相关文章

  • AllenNLP系列文章之四:指代消解

    指代消解是自然语言处理的一大任务之一,它是信息抽取不可或缺的组成部分。在信息抽取中,由于用户关心的事件和实体间语义关系往往散布于文本的不同位置,其中涉及到的实体...

    sparkexpert
  • textgenrnn 文本生成实战

    文本生成是一件很神奇的自然语言处理任务,深度学习给文本生成带来的全新的技术途径,如这篇文章The Unreasonable Effectiveness of R...

    sparkexpert
  • AllenNLP系列文章之三:机器阅读

    Machine Comprehension (MC) models answer natural language questions by selecting...

    sparkexpert
  • 【论文推荐】最新六篇推荐系统相关论文—注意力机制、多任务、协同跨网络、非结构化文本、TransRev、章节推荐

    WZEARW
  • 幸存者偏差与创业鼓吹

    https://www.sonyaellenmann.com/2018/06/survivorship-bias-and-startup-hype.html

    benny
  • Predicting the Future V2更新

    Predicting the Future with Multi-scale Successor Representations

    用户1908973
  • 2017 MCM Problems ABC Notes by magic2728

    建模的小伙伴们,知道你们都在挑灯夜战,为了梦想做最后的努力啦,先为你们的努力喝彩!

    magic2728
  • ImageNet Classification with Deep Convolutional Neural Networks——AlexNet论文翻译——中英文对照

    ImageNet Classification with Deep Convolutional Neural Networks Abstract We trai...

    Tyan
  • 【论文推荐】最新七篇推荐系统相关论文—协同度量学习、SQL-Rank、用户行为与神经网络、隐私价格、贝叶斯、 IoT、序列感知

    【导读】专知内容组整理了最近七篇推荐系统(Recommender System)相关文章,为大家进行介绍,欢迎查看! 1. Collaborative Metr...

    WZEARW
  • 【论文推荐】最新六篇知识图谱相关论文—全局关系嵌入、时序关系提取、对抗学习、远距离关系、时序知识图谱

    WZEARW

扫码关注云+社区

领取腾讯云代金券