‘Prompt engineering’: The ‘boom’ of artificial intelligence creates a new job: expert in talking to the machine

Pau Martí Felip, 23 years old and graduated in Audiovisual Communication, started working two years ago as a video editor in a digital company. Over time, he saw his work change: “That’s how it came to be prompt engineer (request engineer), which is a creative and technological mix of instructing the AI ​​to respond to you,” he says. The new artificial intelligence applications to create images, text and soon video or music need to receive text requests for what we want. That request can range from a simple phrase (“do some 10-minute stretches for people over 70”) to an intricate eight-line instruction that includes details about an image’s colors, backgrounds, or techniques, for example.

“People will need to understand the machine,” says Martí Felip. “It’s like talking to an animal that most people don’t know how to communicate with, we’re learning that language,” he adds. A LinkedIn search for prompt engineer and “Spain” gives only six profiles in total and one job offer. The offer is precisely from Martí Felip’s company, Raona, which is dedicated to digital transformation in companies and has more than “240 collaborators”. This unique job offer for “request engineers” in Spain is proof of its exploratory nature. In February an offer of prompt engineer of one start up big in the US with an enviable salary.

The job offer is to help Martí Felip, who is aware that he is looking for candidates in deserted land. “At the moment, most resumes don’t have AI skills. If you ask four questions about ControlNet or something more advanced, you see it,” he says. But he is willing to settle: “Someone who wants to learn and is very creative is worth it anyway.” Although he seems striking, Raona is not looking for more computer scientists now: “We are full of programmers but we see that we need more creative people.”

The actual appearance of petition engineers has sparked a debate about their feasibility and future. These are difficult prophecies and even the most skeptical admit two things: it is impossible to foresee anything in this field today and, in the short term (two or three years), such work will be necessary.

“It’s not that the offers of prompt engineering They look like smoke to me,” says Javi López, an investor in startups and disseminator on AI issues (known in networks as @javilop). “They seem temporary to me. During a transition period, many will need specific people already trained. But in two or three years it will be trivial and there will be no concept of prompt as such. Even a 5-year-old child will be able to ask a machine for things, out loud. He will tend to zero the difficulty of being an engineer prompts”, he assures.

An illustrated novel

But López himself has so far obtained more than 27,000 euros of financing of 612 patrons for an illustrated novel with artificial intelligence that includes a guide on getting started and descriptions (prompts) that you have used. That figure multiplies López’s initial forecast by almost five; there is interest in exploring this world. Perhaps one day the requests will be simple, but today López’s examples include bits like this string (roughly translated from the original English): “Ultra-realistic, low angle, on a wooden table in a beautiful kitchen, Canon 5D, DSLR, portrait of 50mm, DOF, V-ray rendering, 8k, ray tracing, Goldenhour lighting, uplightrigid edge lighting”.

López’s skepticism comes from his belief that professionals in each field will know how to handle intelligent machines, a new “engineer” will not be needed: “There were also Excel experts and now all accountants are expected to know the tools of their work . In the end, what will be left is not an engineer prompts, but who was already a designer or artist before. Although the barriers to entry are going to be lowered: it will no longer be necessary so much technique (drawing, lighting, photography), but rather direction and selection”, he affirms.

Pablo Moreno-Muñoz, a researcher at the Technical University of Denmark, also believes that the models will tend to simplify, at least in three ways: “One, training the tools with more data (images, text); two, hours of engineering to build models with greater capacity (number of parameters, size of neural networks); and three, training time and money spent on supercomputers, where AIs find more and more relationships between data, which then allow them to generate better results from them. prompts”.

The key distinction in this debate boils down to a simple question: will AIs be able to understand what we mean like Google search does today, or will the complex needs of a future video, image, tune or story still require specialists? It may also be that a simple use of these tools coexists with something more sophisticated: today you can search on Google or be an SEO expert, and you can use Photoshop or simply apply a filter.

The programmer Simon Willison is an advocate of engineers prompts. He believes that they will need, in addition to communication skills, something from all these disciplines: linguistics, understanding how the language works deep learning that moves these models, psychology, art history, computer security and philosophy. “How can we teach a language model the difference between truth and fiction? What is, in fact, the truth?” Willison wonders.

Silviu Pitis, a researcher at the University of Toronto, also sees a clear future for this profession: “As the models become stronger, we will still need humans to interact with them, teach them to communicate.”

How to train for it

Martí Felip’s specific task is twofold: helping engineers to test the model they are programming from, for example, ChatGPT and then using it to obtain optimal results or helping the client to take advantage of it.

Training is another key issue. Martí Felip misses having learned some code at the university. But most of his training has been autonomous: “I have always been interested in AI. I have been learning with tutorials on YouTube, Twitter accounts and TikTok. In college we should have done a bit more code. I know programming basics, but I miss some more advanced Python. Although it is interesting to have the creative part for the design”, he explains.

Jessica Gutiérrez, an administrative assistant from Gijón, is another of the six people who added “prompt engineer” to his LinkedIn bio. She is an administrative assistant but is dedicated to writing for web pages. The step was almost obligatory: “Now it takes me much less time, if you are not learning to generate text, obviously your work will become obsolete.” copy”, he assures. “I recognize that, four months since I started researching and soaking myself every day, in the end you find an ally in artificial intelligence, although I have come a long way to position myself as early as possible,” he admits.

His training has also been on the internet: “He has been self-taught, trial and error, based on watching videos on YouTube, on Twitch,” he explains. She now sees the future as an open field: “There is a lot ahead to lay the foundations for job profiles. Participating in the ecosystem is very useful to explore business and educational possibilities ”, she assures, although those around him do not see it clearly: “People laugh. They don’t think this has a professional future. And they don’t think the tool will generate everything I ask for. Between laughs I made the book”. Gutiérrez refers to the fact that he has prepared a recipe book where the text and illustrations are all made with artificial intelligence: “I got the recipes in one afternoon. It took me longer with the illustrations ”, he details. If it works well for him, he will continue with a children’s one.

And it’s done without code

One of the wonders of this engineering is that knowing code is secondary. A detail that lowers the access barriers. Andrej Karpathy, respected programmer who led Tesla’s AI and just rejoined OpenAI (creators of ChatGPT), tweeted: “The hottest programming language is English.”

It is possible, says Ignacio Peis, a researcher at the Carlos III University, “to qualify as new programming language into English, since a few lines of text generate programming code routines. You might think that this is a new level of abstraction. However, the programming relationships between the spoken language and the generated code are not defined, they are not universal. It can be verified that the same text entry can generate different codes, since we are talking about probabilistic models”, he explains.

English, therefore, does not always work perfectly the same as a programming language. But that does not prevent it from having a run as a new way of relating to the machine: “In a certain way it is programming”, sums up Pitis. “As a requirement for strong AI, computers must be able to communicate with humans through natural language. If said interface becomes strong enough, we can teach them by talking to them,” he adds.

Although English is a possible programming language, machines can beat humans at that as well. A recent academic article proves it with automated requests when trying to improve a human request, and it succeeds if in some questions, it is added at the end: “Think about it step by step”. In the study they assure that the best formula would be something like this: “We are going to solve this step by step to make sure that we have the correct answer.” The race to achieve the perfect request has only begun, says Gutiérrez, who does the tests apart from him: “You have to be very sure of the prompt. I write it aside two or three times and if I don’t have all the details, I keep adding and I don’t ask the machine until I have it complete”.