Generative AI

The evolution of generative AI

While it has taken the world by storm, generative AI is not new – it’s built from technologies we’ve used for decades, including AI, machine learning and statistical methods. The origins of generative AI could be traced farther back, but we’ll start with 1966 and a chatbot named ELIZA.

Joseph Weizenbaum, who built ELIZA, designed it to imitate Rogerian psychotherapists who mirror what the patient says. ELIZA used pattern matching to accomplish this feat. ELIZA was one of the first programs to attempt the Turing Test – an imitation game that tests a machine’s ability to exhibit intelligent behavior like a human. 

As methods of analyzing unstructured text data evolved, the 1970s through the 1990s saw growth in semantic networks, ontologies, recurrent neural networks and more. From 2000 through 2015, language modeling and word embedders improved, and Google Translate emerged.

In 2014, Ian Goodfellow and colleagues developed the generative adversarial network (GAN), setting up two neural networks to compete (i.e., train) against each other. One network generated data while the other tried to determine if the data was real or fake. Transformer models were introduced in 2017. They included a self-attention mechanism that allowed them to weigh the importance of different parts of the input when making predictions. Architectures like BERT and ELMo became popular, too.

Generative pre-trained transformer (GPT) models appeared next, with the first GPT model arriving in 2018. This generative model was trained on large quantities of text data from the internet. With 117 million parameters, it began generating text similar in style and content to the training data. By 2023, large language GPT models had evolved to a point where they could perform proficiently on difficult exams, like the bar exam.

Generative AI in today’s world

How generative AI works

Some popular examples of generative AI technologies include DALL-E, an image generation system that creates images from text inputs, ChatGPT (a text generation system), the Google Bard chatbot and Microsoft's AI-powered Bing search engine. Another example is using generative AI to create a digital representation of a system, business process or even a person – like a dynamic representation of someone’s current and future health status.

There are three main types of generative technologies (digital twins, large language models and synthetic data generation).

Many other technologies enable and support generative AI:

An algorithm is a list of step-by-step instructions designed to accomplish a specific task or solve a problem. Many computer programs are a sequence of algorithms written in a way the computer can understand. As algorithms begin to supplement or replace human decisions, we must explore their fairness and demand transparency into how they’re developed.

Artificial intelligence makes it possible for machines to learn from experience, adjust to new inputs and perform humanlike tasks. AI often relies heavily on deep learning and NLP. Through such technologies, computers can be trained to accomplish specific tasks by processing large amounts of data and recognizing patterns.

Data management is essential for ensuring trusted, ethical and bias-free outputs. It’s particularly critical for AI, machine learning tasks and LLMs that are trained on huge data sets then used to understand and generate content.

Deep learning is a subset of machine learning that trains a computer to perform humanlike tasks, such as recognizing speech, identifying images and making predictions. It improves the ability to classify, recognize, detect and describe using data. Deep learning models like GANs and variational autoencoders (VAEs) are trained on massive data sets and can generate high-quality data. Newer techniques like StyleGANs and transformer models are good at creating realistic videos, images, text and speech.

Machine learning is a method of data analysis that automates analytical model building. It’s a branch of artificial intelligence that trains a machine how to learn. Machine learning is based on the idea that systems can learn from data, identify patterns and make decisions with minimal human intervention.

Natural language processing is a branch of artificial intelligence that helps computers understand, interpret and manipulate human language. NLP draws from many disciplines, including computer science and computational linguistics, to fill the gap between human communication and computer understanding.

Neural networks are computing systems with interconnected nodes that work much like neurons in the human brain. Neural networks use algorithms to recognize hidden patterns and correlations in raw data, cluster and classify it, and continuously learn and improve over time.

Reinforcement learning is when an algorithm discovers through trial and error which actions produce the greatest rewards. A machine learning model, reinforcement learning relies on a reward signal for its feedback mechanism as it gradually learns the best (or most rewarding) policy or goal. It’s often used for robotics, gaming and navigation.

Discover how organizations are using GenAI tools

Many organizations are adopting generative AI, with banking and insurance industries leading the way in adoption for daily business operations. This infographic highlights some early successes – such as managing risk, improving employee satisfaction, saving on operational costs and achieving higher customer retention.

Next steps

See how GenAI solutions can augment human creativity and endeavors.