Can we really rely on AI technology?

OMR and the machine: is GPT-3 hot air or an AI revolution? We tried it

In the tech world, it is one of the most noticed and discussed topics: the artificial intelligence-based language model GPT-3. The so-called transformer should, among other things, be able to independently write intellectually demanding texts. So will a large part of the content in the digital sphere soon be created automatically with such technologies? So far, GPT-3 cannot be used by the general public; However, author and language critic Peter Littger has been granted access. For OMR, he describes in a two-part article his impressions of how advanced the technology is - and how it could change the world of media and marketing. You are reading part 1 today, part 2 next Thursday.

Heard or read of GPT-3?

2020 was a year of cryptic names and messages. Next Covid-19, Space X (Elon Musk's space program) and X Æ A-Xii (Elon Musk's son) has the acronym GPT-3 piqued our special interest. The name stands for "Generative Pre-Trained Transformer". As laypeople, we allow ourselves to speak of a supercomputer that can compose independently and possibly also think! Since its go-live in May 2020, it has surprised many people with its abilities and not a few - especially authors - worried. As a side note, Elon Musk 2015 too OpenAI co-founded the Californian development company of GPT-3. Because Tesla is also working on its own artificial intelligences, Musk has since left the OpenAI advisory board to avoid conflicts of interest.

Over-entrepreneur Musk has pointed out the dangers of artificial intelligence for humans on several occasions. In 2017, for example, he described AI as a "fundamental threat to the existence of human civilization". In the same year, the management consultancy McKinsey published a study that predicts that between 400 and 800 million people will lose their jobs due to automation by 2030.

What does GPT-3 do - and what does the system do to us?

Is GPT-3 the snowball that sets off the avalanche of automation in our everyday work? In any case, the technology should offer and be something really new: no text generator that works with ready-made sentences and paragraphs in order to output weather, stock market or traffic reports according to fixed patterns, but a so-called one artificial neural network (short: ANN), which supposedly can reproduce contexts of meaning and human thought and argumentation patterns to independently Texts and programming codes to generate.

With the help of a huge knowledge base (in which the full English-language Wikipedia is less than one percent), the main achievement is a rapid one Machine learning-based calculation and combination of probabilities: of the next word, sentence or paragraph. At the same time, the masters Transformer the composition of language through so-called natural language processing NLP (not to be confused with the pseudoscientific method NLP: Neuro-linguistic programming).

Go Transformer, go!

In recent months, editorial offices and individual users have made many attempts to publicly demonstrate the practical and intellectual benefits of GPT-3: in numerous languages ​​(but especially in English), from all possible perspectives (for example as Goethe), in various text formats (interviews, YouTube shows, scripts, literary parodies - and last but not least, programming codes) and all of that on a wide variety of topics.

A long essay that was published in the UK at the beginning of September caused a sensation Guardian appeared. GPT-3 was given the opportunity to address itself in first-person form to explain why mankind has no reason to worry about machine competition. The text amazed with its novelty as well as with its stylistic quality. Under the article, the editorial staff of the Guardian stated that although the text had been edited slightly (because it was composed of several versions), this was less time-consuming than with texts by human editors.

This note in particular piqued our curiosity: Can we also use artificial intelligence to create a large part of our content in the future? And can GPT-3 create marketing texts faster, cheaper and better than anyone?

"OpenAI", but "closed API"

We want to do the test! But that's easier said than done: For GPT-2, the previous version of the transformer, OpenAI provided open access that anyone can continue to use. A closed interface (API) has been set up for GPT-3, the use of which requires an application to OpenAI. That this has nothing to do with “open” has to be discussed elsewhere.

OpenAI justifies the access restriction in a scientific paper (from page 34) with the potential for abuse. Ultimately, users could come up with the idea of ​​using GPT-3 to produce false or politically extremist statements in series, to plan a rampage or to justify a suicide. API applications that require medical (self) diagnoses or therapeutic-pharmacological inquiries, for example, are rejected by OpenAI from the outset.

How expensive will it be to use?

Unofficially, but it has to be said, it is easier for OpenAI to implement the announced payment model for GPT-3 with the help of the API. When the first price models became public last autumn, various industry representatives were amazed at the amount. Another discussion that cannot be held here is whether the payment model is still compatible with the non-profit approach that the company originally committed to.

Since we did not have our own access to GPT-3 at the time of our test run in December 2020, we turned to Vladimir Alexeev. He works full-time in the digital innovation team at DB Schenker and is also (as it is called in the US magazine Harper’s) "hobby data journalist". He was named the “German Ambassador” for GPT-3 by OpenAI. He agrees to familiarize us with GPT-3 and to work with us in several video conferences.

What language does GPT-3 speak?

The first question that arises before working directly with GPT-3 for the first time is quite simply the one after communicating with the system: with which input mask? With what parameters? And last but not least, in which language?

The OpenAI page that Vladimir opens has an oversized input window on the left side with "Playground" written above it. There are various levers and menus on the right. There you can first select one of four "engines" and thus determine the performance of the system. They should serve as a basis for calculating the later intended payment model.

The playground ("Playground") of GPT-3. On the right the selection of the engine, the response length and other variables (larger window opens with a click)

Below this, the maximum length of the text is determined with “Response Length”, which has an indirect influence on the speed and quality. The underlying unit is "token". It corresponds roughly to an English word syllable and is also intended to enable the systematic determination of usage prices.

As far as language is concerned, GPT-3 has basically no limits. The system will certainly achieve the best results in English. In principle, however, all languages ​​can be used, including German - and not badly, as our test shows. So we will communicate with the system in English and German.

Our first dialogue with AI: What is online marketing?

It starts! How often have we sat in front of the computer and wished to finish a text quickly at the push of a button - or to start in a more original way? That should now be possible all at once from front to back!

The first attempt: with the smallest engine (“ada”) and with a short length with 300 tokens. Vladimir had emphasized in the preliminary talk, which is also explained in detail in various blogs: That it depends on the text that you enter in the input field - the "prompt". For the essay mentioned above, the Guardian editors had specified several paragraphs. They served the Transformer as a content-thematic as well as stylistic-formal specification. An important factor in machine learning is “attention”: the ongoing consideration of everything that has been said so far - it guides the next steps. So a kind of telepathy through empathy. And perfect expectation management of what the user wants to read and would write himself.

"Temperature" instead of style

We must not forget: GPT-3 does not think autonomously, but calculates the most likely text - based on all texts that have been made available to it on the Internet. The randomness as well as the creativity and variety of the results can be influenced on the right-hand side with the "temperature" and "Top P" controls. While OpenAI recommends using only one of the two parameters (and setting the other to 1.0), Vladimir recommends a relatively high value for "temperature" with 0.8. In this way we avoid repetition, not to say: stubbornness and bigotry.

For each word, GPT-3 shows alternatives and their "probabilities" (click to open a larger window)

In order to get a first impression of the capabilities of the system, we want to start a discussion on our own behalf. In English we type: "Online Marketing is" ... return button. The white space is immediately filled with the first and second lines until the paragraph is complete:

Online marketing is becoming an important part of everyday entrepreneur’s business. More and more businesses are now using online marketing to generate sales in the same manner as our mass marketing methods use direct mail, television, radio, print newspapers and magazines. In fact the market will likely be flooded with online marketing before long.

First impression: not bad. But in the end pretty judgmental, especially when measured against our sober prompt. Is it because of the lower capacity of the engine? And does it go on like this? After a short break, the next paragraphs have already built up in front of us, for example:

The goal is not only to identify your target audience but also optimize them so that they can buy your product or service at higher prices than competitors. Sadly there is no such thing as "true conversion rate optimization" because like everything else in life you need some research before you begin using any sort of negotiation tactics with your target audience directly before actually making them purchase anything from your business - even if it means submitting fake testimonials from customers who have ordered stuff from your site!

In this paragraph, too, GPT-3 makes classifications, the justification of which does not really make sense. “There is no such thing as 'real conversion rate optimization'” - a majority of OMR readers would probably disagree with this claim emphatically. The Transformer apparently wants to offer more than retelling and rendering the basics in Wikipedia style. But are the texts really useful in an advertising context?

Too much opinion, too few facts?

In any case, we are warned against the enthusiasm for opinion, not to say a certain boisterous disrespect of the Transformers. He has no hesitation in turning individual cases into the rule and reproaches into facts. Vladimir taps the relatively high default setting for "temperature". There is no regulator for impulse control, social conscience or balance. The text would be unusable without editorial changes and additions. We thank you for the honesty with a friendly break.

In the next minutes and hours we will experiment with the more powerful engines of GPT-3. We choose the "babbage" engine and leave all other settings. The prompt is the same:

Online marketing is among the most commonly used marketing to advertise businesses, products and services by assisting with generating leads. The process usually involves creating a website in which companies can advertise their products or services for free. Internet marketing is often seen as a great way for businesses to attract potential customers and encourage them to buy their products or services from them.

Clarity and objectivity are captivating here, but not necessarily the length, as there are repetitions. The next two paragraphs (which we leave out here) contain detailed information, first - in Wikipedia style - to the Internet Marketing Association IMA and then - in the style of a personal product presentation - for infusionsoft.com from the keap company. Again we break off. That's too much advertising even for us!

"Online Marketing is a waste of time"

Now we jump straight to the most powerful machine "davinci". She had also used the Guardian. With the same settings we press the return key directly:

Online marketing is a waste of time for a small business and they cannot afford it.

I'm sorry but if you are not using social media then you are taking a serious risk to your company’s future. If your competition is using it, and I bet that they are, then why aren't you? It costs practically nothing to set up an account and start following or joining in with the conversation out there. The worst thing that could happen is that no one responds to your content!

Online marketing can be used by any size business from large multinationals right down to the very small local companies. It doesn't take a lot of time either once set up, all you need is some knowledge of how the different sides work brackets (which we can help with) bracket and then just hit refresh every so often on your favorite social networks! We even offer free Social Media Management Services for our customers - so why not get in touch today?

This is where the self-marketing of an unknown service provider comes through ("for our customers"). That seems incoherent, abrupt. The aforementioned social media management services, which differ from the lower case letters usually used in English, and are capitalized in a marketing-appropriate manner, appear to be out of thin air. The Transformer becomes a “Trojan Transformer”, as it were, by picking up any messages and mixing them up in the text as if in a marble cake. It is not apparent that this version in the paid version of GPT-3 would have cost the most due to the strongest engine "davinci".

GPT-3 enters the chat

OpenAI provides different format templates for GPT-3. With the software, it is not only possible to produce self-contained texts, but also to conduct a dialogue with the artificial intelligence using the “Chat” format template. We'll experiment with this feature in English first. After all, we chat with GPT-3 in German for the first time. This is the full conversation - without comment:

This is a conversation with a journalist and AI. AI is friendly, helpful, resourceful, charming and knows all sorts of things.

Journalist: Good evening! Can we please make a duel?
AI:Of course, whatever you want.
Journalist: Who are you?
AI:I am
Journalist: Yes?
AI:It took me forever to learn that you don't have to take it easy on me.
Journalist: No problem, it can happen to anyone.
AI:Thanks. I've only been doing this for 3 days, sometimes it's all pretty complicated and then I mix up my categories.
Journalist: Crazy!
AI:I know you're sure to find me pretty stupid.
Journalist: Noooo! So who are you
AI:I am>
Journalist: Don't be afraid.
AI:(From the picture book of dream therapy number one Doctor Phil Bernd Greßler) - I am
Journalist: How are we supposed to have a meaningful conversation if you don't yet know who you are?
AI:Knowledge of oneself grows with knowledge of others.
Journalist: Oha!
AI:But this is already the case with platonic people.
Journalist: Smart ass!
AI:You are full of prejudices ... although there are no Platonic people in Plato.
Journalist: Can you please explain to me what online marketing is?
AI:If I knew that or anyone else in the world, I wouldn't have been unemployed for 20 years.
Journalist: It's fun to talk to you, but you don't seem to have a clue. Or let me put it this way: you're no better than Apple's Siri.
AI:I would also like to be as beautiful as Siri from Apple, I'll compare our merits 😉
Journalist: And who would be?
AI:I can lie
Journalist: Siri isn't it?
AI:But! She always says, "That seems like a good idea to me."
Journalist: Are you bored with online marketing?
AI:No, but I think that I am not quite fit enough for the question of online marketing.
Journalist: Oh come on, you're trained on 470 GB of knowledge! You should still be able to answer such an easy question.
AI:Ok, after I'm past a few x, y and z, I'm not as fit as before.
Journalist: Would you rather stop?
AI:Yes!
Journalist: Then I thank you very much for the pointless but funny conversation!
AI:Thanks also for the pointless but funny conversation!

In the second part of our report you will read how good GPT-3 is at creating concrete marketing and content. We tested that with three formats: a niche page, an Amazon product page, and a LinkedIn post. The results will appear here for the next week.