The future of synthetic media

What are synthetic media?

Synthetic media are various media, such as pictures, videos, texts or sound clips, created or modified by artificially intelligent (AI) software. This type of AI software creates realistic examples or modifies existing media to create entirely new media.

It becomes possible to:

• create completely new and believable videos
• swap faces in videos
• reproduce someone’s voice
• Generate credible texts
• erase objects in videos or photos as if they were never there.

Note: all photos in this article were created by AI software.

Artificially intelligent software becomes our creative assistant

With this kind of software it becomes possible to generate new ideas based on existing examples. So no copies, but completely new variations. The photos, videos, texts or sound fragments are created by the “imagination” of the smart software. AI software thus becomes an idea machine.

In recent years, that smart software has taken over more and more human skills from us, such as looking, listening, speaking and reading. And now the first promising next steps are being taken in the area of imagination.

Photomodels, cats and Airbnb

Known are the examples of photo models, people, cats, resumes or Airbnb pages that don’t exist in real life at all. You may also be familiar with deepfake technology where the face of one person is swapped with that of another. And now artificially intelligent software also makes music, based on examples that human artists came up with earlier. This kind of artificially intelligent software is now also capable of composing texts based on existing material. Or to write right-wing extremist manifestos. Some of these texts -originating from AI- are indistinguishable from texts written by humans. There are also digital avatars that effortlessly read out texts in different languages. Synthetic art expressions are also very rewarding. Humans have a secondary role in these…

Creativity for all

Synthetic media democratize the process of creativity and make it available to everyone. People can create their own creative content ever more easily and cheaply, for example produce high-quality videos starring themselves. It will also be possible to have a text generated quickly by an artificially intelligent system. This could be a first draft for an article or an idea for a fun angle. You can also collaborate with this kind of AI software to write completely new stories. The author makes a starter, the machine creates a story and the author polishes it. This allows the author to create new stories faster and easier, which might be much more original without this collaboration. It is already very cheap to create your own content, but the price will only get lower, while the quality goes up. Creativity thus becomes a process that takes place between man and machine.

Example: Synthesia

Take this example: Synthesia. With this software you can create a digital avatar. The only input is text. From this written text, the artificially intelligent software then creates a digital avatar that reads your text.

The appearance of this avatar is almost indistinguishable from the real thing. Only the synthetic voice betrays the difference with a human being of flesh and blood. This type of software ensures the democratization of video production. Video production available to everyone. You don’t need a studio, no microphone, no actors, no cameras. Everything you make happens on your computer. At the speed at which you normally write an email, you then create a video with a digital presenter.

Instructional videos

You can use the software to generate company presentations, create instructional videos for software, or use it for internal communications.

Are you an international company? Then this software translates into 39 languages. Are you a housing association with multilingual tenants? Then this software is ideal for explaining something to your tenants, for example. You can use this AI software to automatically create individual product videos for your webshop. Or as a seller of real estate produce a unique video of each apartment in a complex. Because the source of input is text, you can also easily process first name, last name, department and that kind of data and then make a fully personalized video.

Now video production is often still a static and linear process. You make a video somewhere on location with a collaborator, actor, technician and sound engineer. A time-consuming process. Not to mention: everything has to be right the first time. And then it’s done. Done. In the future this will be different. Video production will become a much more continuous process. Does something change in terms of content? New video. Got a cool angle? New video!

Co-fanting machines

Synthetic media systems create new content and thus create new ideas. They make infinite creative suggestions -for example, when painting a landscape- and create variations on our own creative input. Be it video, text, music and so on. In the future, we will no longer know whether something was generated by a computer, conceived by a human or actually took place in reality.

Existing worlds can be cloned by software in the future into infinite “lookalikes”: a digital virtual world. Ideal for training self-driving cars, for example, or to make the work of game designers easier.

DALL-E: text to image

IN January 2021, OpenAI introduced a new software tool: DALL-E. The artificially intelligent software creates images based on written text. It creates completely new (!) pictures based on a sentence. The software has learned to establish the relationship between words and visual output. Very spectacular!

There are many brilliant examples in OpenAi’s blog:

• a store front that has the word ‘openai’ written on it
• an armchair in the shape of an avocado
• an illustration of a baby daikon radish in a tutu walking a dog

Artificial Creativity

It is not difficult to be excited about this new landscape of possibilities. We as humans will increasingly be able to interact with artificially intelligent machines in the areas of creativity and innovation. In some areas, they will greatly enhance the creative process by generating new ideas at speed. Some ideas may seem silly or even hallucinatory at first, but sometimes they are surprisingly innovative and inspiring. And that includes not only visual input, but also text and language.

Super-sophisticated auto-complete software

A well-known example of this kind of text technology is the language model GPT-3. It is an AI system that has been trained to generate credible texts independently. In fact, the GPT-3 system can do one thing very well: predict the next word in a given sentence. As a result, GPT-3 can produce complete texts that resemble texts written by a human. In doing so, by the way, the computer system has no knowledge of the “meaning” of words, but is especially good at making word predictions. So it’s actually a super advanced auto-complete function, like you know it from your phone.

The Future of Synthetic Media

In the future, these kinds of artificially intelligent systems will write complete email messages for us. Then we will only have to read through those messages before approving and sending them. It is even conceivable that, in the somewhat distant future, artificially intelligent systems will respond on our behalf to easily answered e-mails. This still sounds futuristic, but the quality of that kind of GPT-3 autocomplete software is improving by leaps and bounds. And who knows, maybe in the future GPT3-like software will make completing our sentences in emails, blogs and posts on social media feel very natural. Translating large chunks of text will become almost perfect in that case in the not too distant future and will take place almost in real time. Also, the writing of a summary by AI software is certainly a possibility.

GPT-3 software is already a good “hypothesis creating machine. For example, on this website it creates ideas for startups. Some very silly, but other ideas are brilliant again. Synthetic media systems are then like co-fantasy machines. And there are many more areas where we are going to get help from artificially intelligent software that will create, invent or adapt content. Sometimes with the help of people, sometimes fairly autonomously.

Synthetic speech

By the way, we are not only seeing generative AI software being used in image and text generation, but also in audio. Not only do we see generative AI computer systems creating music, but even the human voice is being generated by AI. Some companies have already succeeded in generating synthetic voices that are indistinguishable from real ones, such as Google’s Wavenet. The robotic voice has long since ceased to be tinny and wooden, but virtually indistinguishable from the real thing. It is also possible to clone your own voice.

Applications of synthetic speech and voice cloning: Spotify advertising, for example, can be quickly personalized when an advertising voice also uses your name. Newsreaders can read out the news 24 hours a day, as long as they are fed new texts. You can clone the voice of your loved ones so you can talk to them through your smart speaker even after they die. It will be easier to release an audiobook alongside your regular book.

Scenarios

Bad scenario: When this works flawlessly, you can have someone else say anything in an audio excerpt. Libel, reputational damage, blackmail, identity fraud. Rapper Jay-Z, by the way, recently objected to voice cloning videos in which he supposedly rapped Billy Joel’s “We Didn’t Start the Fire. It is logical that voice cloning is making its appearance in the music industry. After all, there are plenty of audio samples from artists to train smart AI systems with. It does raise the question: does someone own his or her own voice? That depends on a number of factors. For example, whether you are a public figure or not. And whether a video is meant for satire, whether you want to make money with a fake album or use it to put someone in a bad light.

Personalized product placement

Synthetic media will make it possible in the future to personalize movies, videos and games based on the user. Each individual user will then be shown specific images, matching his or her profile. Do you like Gucci clothing? Then you’ll get to see that in a video game. Are you more into sustainable Kuyichi clothing? Then you might get to see garments from the latest collection in a video you watch. Do you like BMW? Then the car used in the latest episode of “Homeland” might be – especially for you – a BMW. Of course, this is future music, but synthetic media makes individualized product placement possible. The quality of content creation is increasing, while the price of making it is becoming very low.

Cats created by artificial intelligence

Digitally cloned celebrities

Artists and celebrities are creating digital versions of themselves with synthetic media software. A digital clone or avatar that can represent them online. Actors can make their digital appearance in a movie without having been physically at the shoot. And because their voice is cloned, it is also possible to use it somewhere else in the world to produce voice-overs for advertising commercials. A digital chatbot simultaneously answers online questions posed by fans, but in the tone of voice of the celebrity himself.

The digital voices of celebrities are also likely to be “hired” by commercial companies to read out press releases, content reports or web texts with their distinctive voice tone. Voice cloning of celebrities as a commercial industry is very interesting because it only requires an investment of time. After that, celebrities can lend their voice to third parties infinitely often and for a long time, without having to invest any more time in it themselves, while collecting the revenue.

Digital avatars

Digital avatars are digital versions of people. They exist only in the virtual world. They are, especially on the outside, a credible reflection of human appearance. In the future, their quality will only increase. Not only in appearance and the way they move, but also in speech and conversation.

Everywhere we request information, digital avatars may be in the future. They’ll tell us bus, train and plane departure times. And they will read the latest news to us 24 hours a day, both online and on television. Even breaking news can be brought to us by these speech robots. At any time of day, 24/7, they read us the weather forecast, take our orders at fast food restaurant and in some places even do a brief intake. It doesn’t have to matter where you are as a consumer: from the comfort of your home, on your smartphone or at the location itself.

Frequently asked questions

Digital avatars can be trained to answer the “frequently asked questions” that come in to customer service in large numbers. A big advantage: these virtual ’employees’ are scalable and can easily handle ten thousand calls per minute. They are friendly, service-oriented, they always follow the protocols. They are never tired or grumpy, they avoid conflict, and they are learning more and more empathy.

The potential market for this kind of software, where digital avatars answer the most frequently asked questions for a company, is immense. When this software works well and consumers get used to the idea of talking to a machine, this solution will become commonplace. As a result, staff whose jobs require them to answer simple customer questions could be partially or entirely replaced by this software.

Digital avatars will also be found in the education and training industry. Has your employer introduced new software? A Digital Avatar explains it to you in a personalized video with your own name, function and smoothly spoken in your own language.

Create and customize

Synthetic media offer many advantages and opportunities: deceased actors, for example, can be “brought back to life” more easily than ever in new movies with deepfake-like technology. (When people talk about deepfakes, they often mean the videos where faces are switched).

Actors no longer have to redo certain scenes and dialogues can be adjusted afterwards. Advertisements can easily be adapted to multiple language areas, so that it is not noticeable that the Dutch spoken advertisement is dubbed from the German. Stuntmen need to do less dangerous work in movies, and historical figures can appear digitally in the present, making education more interesting for students.

Synthetic media is a new revolution in the media landscape. After the rise of mass media, the Internet, social media / web 2.0, this is the fourth media revolution. Don’t wait too long to seize the opportunities, I would say.