AI Glossary

Unpacking the black box: Demystify the alphabet soup of AI with this essential resource​

Definitions, examples, and context – your one-stop guide to mastering the vocabulary of the future.

AI (Artificial Intelligence)​

Artificial Intelligence (AI) refers to the simulation of human intelligence in machines that are programmed to think and learn. It's a broad field encompassing everything from simple automation to complex problem-solving abilities.

Example: AI is used in virtual assistants like Siri and Alexa, which understand and respond to voice commands, making everyday tasks more convenient.

ML (Machine Learning)​

Machine Learning is a subset of AI focusing on the development of systems that can learn and improve from experience without being explicitly programmed. It plays a crucial role in creating intelligent systems that adapt over time.

Example: Netflix's recommendation system is a product of ML, where the platform suggests movies and shows based on your viewing history.

Deep Learning​

Deep Learning is a subset of ML based on artificial neural networks with representation learning. It allows machines to process and learn from vast amounts of data in a way that mimics human thought processes.

Example: Deep learning powers advanced image recognition systems, such as those used in self-driving cars to identify obstacles and navigate roads.

Neural Network​

A Neural Network is a series of algorithms in AI designed to recognize patterns by interpreting sensory data through a kind of machine perception, labeling, or clustering of raw input.

Example: Handwriting recognition in tablets, where the device identifies letters and words written with a stylus or finger.

Supervised Learning​

Supervised Learning is a type of ML where the algorithm is trained on a labeled dataset, which means the output is known, enabling the model to learn over time to produce the desired outcome.

Example: Email spam filters are an application of supervised learning, where the system learns to identify spam from non-spam emails.

Unsupervised Learning​

Unsupervised Learning involves training an algorithm on a dataset without predefined labels, allowing it to identify patterns and relationships on its own.

Example: Market basket analysis in retail, where unsupervised learning helps determine products frequently bought together.

Reinforcement Learning (RL)​

Reinforcement Learning is an area of ML where an agent learns to make decisions by performing actions and receiving rewards or penalties, learning through trial and error.

Example: Video game AI, where characters learn to navigate and improve their strategy based on the outcomes of their actions.

RLHF (Reinforcement Learning from Human Feedback)​

RLHF is a technique in ML where an agent learns from human feedback, integrating human judgments into the learning process to refine its actions and decisions.

Example: Language translation programs that improve their translations based on user corrections and feedback.

Constitutional AI (CAI)​

Constitutional AI is a framework where AI systems are designed to learn and operate within the boundaries of predefined rules or 'constitutional' guidelines, often utilizing other AI systems for feedback and learning, rather than direct human input.

Example: A content moderation AI that learns to filter online content based on a set of ethical guidelines and rules, continually refining its understanding and application of these rules through feedback from other AI systems.

NLP (Natural Language Processing)​

Natural Language Processing (NLP) is a branch of AI that focuses on enabling computers to understand, interpret, and respond to human language in a useful way.

Example: Voice-activated GPS systems use NLP to understand spoken directions and provide driving instructions.

Chatbot​

A Chatbot is a software application that uses AI to conduct a conversation via auditory or textual methods, often used in customer service to handle inquiries without human intervention.

Example: An online retail chatbot that assists customers with tracking their orders and answering FAQs.

GPT (Generative Pre-trained Transformer)​

Generative Pre-trained Transformer (GPT) is an advanced AI model designed for understanding and generating human-like text. It's a type of neural network known as a transformer, which is trained on a large dataset of text to predict the next word in a sentence. This training allows it to generate coherent and contextually relevant text based on the input it receives.

Example: ChatGPT, which uses GPT, can write essays, answer questions, and even generate creative stories based on the prompts it is given, showcasing its ability to understand and produce human-like language.

Bias in AI​

Bias in AI refers to the unintentional prejudice in the decision-making process of AI systems, often reflecting biases in training data or algorithms.

Example: A job application screening tool that inadvertently favors candidates from a certain demographic due to biased training data.

Algorithm​

In the context of AI, an algorithm is a set of rules or instructions designed to perform a specific task or solve a particular problem, forming the basis of intelligent systems.

Example: The Google Search algorithm, which sorts through billions of web pages to find the most relevant results for a query.

Data Mining​

Data Mining is the process of analyzing large datasets to discover patterns, correlations, and insights, often using statistical techniques and AI.

Example: Supermarkets analyzing transaction data to understand shopping patterns and optimize stock levels.

Big Data​

Big Data refers to extremely large datasets that may be analyzed computationally to reveal patterns, trends, and associations, especially relating to human behavior and interactions.

Example: Social media platforms analyzing user data to tailor advertisements and content recommendations.

Robotics​

Robotics is a field that combines AI with mechanical engineering to create robots capable of performing tasks that are either too dangerous, repetitive, or difficult for humans.

Example: Autonomous robots used in manufacturing for assembling products with precision and efficiency.

Algorithmic Fairness​

Algorithmic Fairness is the pursuit of creating AI algorithms that make unbiased decisions and treat all groups equitably, regardless of race, gender, or background.

Example: Developing a credit scoring AI that does not discriminate based on demographic factors.

Transfer Learning​

Transfer Learning is a method in AI where a model developed for one task is reused as the starting point for a model on a second task, enhancing efficiency and reducing the need for extensive data.

Example: Using a pre-trained image recognition model to start developing an AI for diagnosing diseases from medical imaging.

Explainable AI​

Explainable AI involves creating AI systems whose actions and decisions can be easily understood by humans, making AI more transparent and trustworthy.

Example: A loan approval AI system that provides explanations for its decisions, allowing loan officers to understand why certain applications are approved or rejected.

GANs (Generative Adversarial Networks)​

GANs are a class of AI algorithms used in unsupervised machine learning, implemented by a system of two neural networks contesting with each other in a zero-sum game framework.

Example: Creating realistic computer-generated images or videos that can be used in video games and film productions.

AI Ethics​

AI Ethics are the moral principles and guidelines that govern the creation, implementation, and use of artificial intelligence. These principles ensure that AI technologies are developed and used in a way that is beneficial, fair, and does not cause harm to humans or the environment.

Example: A company developing an AI for loan approvals adopts AI Ethics by ensuring their algorithm does not discriminate based on race, gender, or other personal characteristics.

Responsible AI​

Responsible AI refers to the practice of designing, developing, and deploying AI with accountability, fairness, transparency, and ethical considerations at its core. It involves ensuring that AI respects human rights, operates reliably, and is used in a way that benefits society as a whole.

Example: A facial recognition software company conducts regular audits and allows third-party reviews to ensure their AI does not exhibit biased behavior against certain groups of people.

Human in the Loop​

Human in the Loop is a framework where human judgment is integrated with AI systems. In this setup, humans are involved in critical decision-making processes, especially in areas where AI's judgment may be limited or needs supervision.

Example: In a medical diagnosis AI, doctors review and validate the AI's diagnoses to ensure accuracy before any treatment is advised.

Overfitting​

Error in machine learning where it functions too closely to the training data and may only be able to identify specific examples in said data but not new data.

Example: A stock prediction model is trained on five years of data from a specific stock market. It performs exceptionally well on historical data but fails to predict future trends accurately because it has overfitted to the patterns in the past data.

Underfitting​

Underfitting occurs in machine learning when a model is too simple to capture the complexities and patterns in the data. As a result, it performs poorly on both training and new data, failing to make accurate predictions or classifications.

Example: A spam filter that only uses email sender names to identify spam might underfit, as it ignores the content of the emails, leading to many spam emails being missed and some legitimate emails being incorrectly marked as spam.

Vector Database​

A Vector Database is a type of database designed to efficiently store and query vectors, which are arrays of numbers that represent complex data in a format suitable for machine learning and similar algorithms.

Example: A recommendation system uses a vector database to store user and product information as vectors. The system can then quickly find products similar to a user's interests based on the cosine similarity between vectors.

Retrieval Augmented Generation (RAG)​

Retrieval Augmented Generation is a technique in AI where a generative model, like a language model, is combined with a retrieval system. The retrieval system fetches relevant information from a database or corpus, which the generative model uses to enhance its responses or content generation.

Example: A chatbot uses RAG to answer complex questions. When asked about current events, it retrieves recent news articles to generate an informed response.

LLM (Large Language Model)​

A Large Language Model (LLM) is an advanced AI model designed to understand, interpret, and generate human language. These models are trained on vast amounts of text data and can perform a variety of language-related tasks.

Example: GPT-4, an LLM, assists users in writing creative stories, answering questions, and even coding by understanding and generating human-like text based on its training.

OCR (Optical Character Recognition)​

OCR is a technology that converts different types of documents, such as scanned paper documents, PDF files, or images captured by a digital camera, into editable and searchable data.

Example: A library uses OCR to digitize ancient manuscripts, making the texts searchable and accessible online.

Mixture of Experts (MoE)​

Mixture of Experts is an AI model architecture where several different neural network models (the 'experts') are trained to specialize in different parts of a dataset. The system then decides which expert to use for a given input.

Example: In language translation, an MoE system might use different expert models for different language pairs or complexities of sentences.

Model​

In AI, a model refers to the specific structure, trained on data, which makes predictions or decisions based on input. It's an output of machine learning processes.

Example: A predictive model that forecasts weather conditions based on historical meteorological data.

Reward​

In machine learning, particularly in reinforcement learning, a reward is a signal given to an AI model, indicating how well it performed a given task or made a decision, guiding its future learning.

Example: In a chess-playing AI, winning a game or capturing an opponent's piece might be programmed as a reward to reinforce successful strategies.

Synthetic Data​

Synthetic data is artificially generated data, as opposed to real-world data, used to train AI models. It can help overcome issues like data scarcity and privacy concerns.

Example: Generating artificial patient records to train a medical diagnosis AI without using actual patient data.

Interference​

In the context of AI, interference often refers to the phenomenon where an AI system's performance is impacted by external or unexpected factors during its operation.

Example: A voice recognition system misinterpreting commands due to background noise.

Prompt​

A prompt in AI, especially in language models, is the initial input given to the model to elicit a response or action. It sets the context or question for the AI to address.

Example: Asking a language model to write a poem about the sea; "Write a poem about the sea" is the prompt.

Prompt Engineering​

Prompt Engineering is the skillful crafting of prompts to effectively communicate with and elicit specific responses or behaviors from AI models, particularly in natural language processing.

Example: Refining prompts to a language model to generate more creative or precise responses in a chatbot.

Foundation Model​

A Foundation Model is a type of large-scale AI model that is pre-trained on vast amounts of data and can be fine-tuned for a variety of specific tasks.

Example: GPT-3 is a foundation model that, after initial training, can be adapted for tasks like translation, question-answering, or content creation.

Open Source LLM (Large Language Model)​

An Open Source LLM is a language model whose source code and training methodologies are publicly available, allowing researchers and developers to study, use, and modify it.

Example: BERT (Bidirectional Encoder Representations from Transformers) is an open-source language representation model that can be used for a variety of natural language processing tasks.

Data Augmentation​

Data Augmentation refers to the process of increasing the diversity and amount of data by adding slightly modified copies or creating synthetic data, enhancing the robustness and accuracy of AI models.

Example: In image recognition, data augmentation might involve flipping, rotating, or altering the colour of images to expand the training dataset without needing new images.

Hallucinations​

In AI, hallucinations refer to instances where a model generates false or nonsensical information, often as a result of limitations in understanding context or handling complex inputs.

Example: A language model generating a fictional historical event as if it were true in response to a query about history.

Multimodal AI​

Multimodal AI refers to AI systems capable of processing and interpreting multiple types of data inputs such as text, images, videos, and speech, often integrating these diverse data sources for more comprehensive understanding and response.

Example: A virtual assistant that can understand spoken commands (audio), recognize faces (image), and interpret written questions (text).

FMOps (Foundation Model Operations)​

FMOps involve the management, deployment, and scaling of foundation models, ensuring their efficiency, reliability, and effectiveness in various applications.

Example: Managing and updating a large-scale language model used across different applications in a tech company.

LMOps (Language Model Operations)​

LMOps refers to the specific operational aspects of deploying and maintaining language models, including aspects like model training, updating, monitoring, and performance tuning.

Example: Overseeing the deployment of a chatbot’s language model, ensuring it performs accurately and efficiently in real-time interactions.

Low-Rank Adaptation (LoRA)​

LoRA is a technique in machine learning where an existing model is adapted with minimal changes to its architecture, often by adjusting only a small subset of its parameters, to optimize performance or adapt to new tasks.

Example: Tweaking a pre-trained image recognition model to specialize in identifying specific types of objects with minimal retraining.

Token​

In AI, particularly in NLP, a token is the smallest unit of data processed, often a word, but it can also be a part of a word or a sentence, depending on the context of the task.

Example: In text processing, the sentence "Hello, world!" might be split into tokens like "Hello", ",", "world", and "!".

Fine-tuning​

Fine-tuning is a process in machine learning where a pre-trained model is further trained (or 'tuned') with a more specific dataset, enhancing its accuracy on a particular task or domain.

Example: Adjusting a general speech recognition model to understand medical terminology for use in healthcare settings.

Embedding​

An embedding is a representation of data where elements of similar type or context are mapped to points in a high-dimensional space, often used to transform discrete variables like words into a form understandable by machine learning models.

Example: In word embeddings, similar words like 'king' and 'queen' have representations that are closer in vector space.

Temperature​

In the context of AI and particularly in language models, temperature is a parameter that controls the randomness of predictions by the model. A lower temperature leads to more predictable and conservative outputs, while a higher temperature results in more diversity and creativity.

Example: Adjusting the temperature parameter to generate more varied and creative responses in a conversational AI.

Frequency Penalty​

Frequency Penalty is a parameter in some AI models, especially in text generation, that reduces the likelihood of the model repeating the same information or words too often.

Example: Applying a frequency penalty to prevent a language model from repeatedly using the same phrases in a content generation task.

Presence Penalty​

Presence Penalty is similar to frequency penalty, but it specifically decreases the chances of repeating the same information or context that has already appeared in the ongoing generation.

Example: Using a presence penalty in a story-writing AI to encourage the introduction of new characters and plot elements, rather than revisiting the same ones.

Stop Sequence​

In AI, particularly in language generation, a stop sequence is a predefined signal or set of characters that indicates to the model when to end the generation process.

Example: In a text-generation task, the model may be programmed to stop generating content upon encountering a specific phrase like "[end]".

Transformer​

A transformer is an advanced model architecture in machine learning, particularly in NLP, known for its efficiency in handling sequences of data (like text) and its use of self-attention mechanisms.

Example: The BERT language model, used in natural language understanding tasks, is based on the transformer architecture.

Self-attention​

Self-attention is a mechanism in AI models, particularly transformers, allowing the model to weigh the importance of different parts of the input data (like different words in a sentence) relative to each other.

Example: In a sentence translation task, the model might focus more on the subject of the sentence for a more accurate translation.

Tensor​

In machine learning, a tensor is a multi-dimensional array of numbers, generalizing vectors and matrices to potentially higher dimensions, used to represent data and operations in neural networks.

Example: An image can be represented as a 3-dimensional tensor with dimensions corresponding to height, width, and color channels.

GPU (Graphics Processing Unit)​

A GPU is a specialized electronic circuit designed to rapidly manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. In AI, GPUs are crucial for their ability to handle parallel tasks and large computations efficiently.

Example: Training complex neural networks on large datasets is significantly faster using GPUs because of their parallel processing capabilities.

TPU (Tensor Processing Unit)​

A TPU is a type of application-specific integrated circuit developed by Google specifically for neural network machine learning, particularly using Google's own TensorFlow software.

Example: TPUs are used to accelerate machine learning workloads for Google services like Google Search and Google Photos.

Encoder​

In machine learning, an encoder is a component of a model that converts raw data (like text or images) into a different, often more manageable, format or representation.

Example: In an autoencoder, used for unsupervised learning tasks, the encoder compresses the input into a lower-dimensional representation.

Chain of Verification (CoVe)​

Chain of Verification is a concept in AI, particularly concerning Large Language Models (LLMs), where a response generated by the model is used to validate or verify itself. This involves the model producing an output and then applying a mechanism to assess the accuracy or reliability of its own output.

Example: An LLM generates an answer to a complex question and then checks various sources or its own database to confirm the correctness of its response. For instance, if the model answers a historical question, it might cross-reference its answer with known historical data or texts it was trained on to verify its accuracy.

Chain of Density Prompting (CoD)​

Chain of Density Prompting is a method designed to enhance the comprehensiveness and detail of summaries generated by GPT-4, specifically by incrementally increasing the number of entities (like key characters, places, concepts, etc.) mentioned in the summary without extending its overall length.

Example: In summarizing a complex news article, GPT-4 initially might focus on the main event and key figures. Through CoD, it gradually includes other relevant entities like secondary figures, locations, or related events, resulting in a summary that captures a wider spectrum of the article’s content in the same concise format.

Self-refine​

In AI, self-refine might refer to the ability of a model to improve its performance iteratively, often by learning from its outputs or additional data. However, this term is not standard and might have specific meanings in different contexts.

Example: A language model that updates its parameters based on feedback from users to improve the relevance and accuracy of its responses.

Chain of Thought​

Chain of Thought in AI, particularly for language models, involves the model expressing its reasoning process step by step, akin to how a human might logically break down a problem. This can lead to more accurate and interpretable outcomes.

Example: Solving a math problem by sequentially outlining each step of the calculation, rather than just providing the final answer.

Skeleton of Thought​

While not a standard term in AI, "Skeleton of Thought" could conceptually refer to a simplified or fundamental structure of a model's reasoning process, laying out the basic steps or elements of thought without detailed elaboration.

Example: In answering a complex question, the model first outlines key concepts or steps before fleshing them out in detail.

Zero-shot​

Zero-shot learning refers to the ability of a machine learning model to correctly handle tasks it has not explicitly been trained on, using its understanding from similar or related tasks.

Example: A language model that can translate a language pair it was never specifically trained on, using its knowledge from other language translations.

Generative AI​

Generative AI refers to AI models that can generate new content or data that is similar to but distinct from what they were trained on, often used in creating images, music, text, or other media.

Example: AI creating realistic human portraits that do not correspond to real individuals.

Turing Test​

Proposed by Alan Turing, the Turing Test is a measure of a machine's ability to exhibit intelligent behavior indistinguishable from that of a human. If a human evaluator cannot reliably tell the machine from a human, the machine is said to have passed the test.

Example: A chatbot participating in a conversation where evaluators cannot discern if it is a bot or a human.

Training Data​

In machine learning, training data is the dataset used to train a model. This data is fed into the model so that it can learn and adjust its parameters for performing a specific task.

Example: A set of labeled images (like pictures of cats and dogs) used to train an image classification model.

Sentiment Analysis​

Sentiment analysis is an AI technique used to determine the emotional tone behind a body of text. This is a common task in NLP, useful for understanding opinions, attitudes, and emotions expressed in written language.

Example: Analyzing product reviews to determine customer satisfaction levels.

AGI (Artificial General Intelligence)​

AGI refers to a theoretical form of AI that has the ability to understand, learn, and apply its intelligence broadly and flexibly, akin to human cognitive abilities. It contrasts with narrow AI, which is designed for specific tasks.

Example: An AGI could theoretically perform any intellectual task that a human can, from writing a symphony to solving a complex scientific problem.

Convolutional Neural Networks (CNNs)​

CNNs are a type of deep neural network, most commonly applied to analyzing visual imagery. They are particularly known for their ability to detect patterns and objects in images.

Example: Used in facial recognition systems to identify and verify individuals.

Data Science​

Data Science is an interdisciplinary field that uses scientific methods, processes, algorithms, and systems to extract knowledge and insights from structured and unstructured data.

Example: Analyzing large sets of customer data to identify purchasing patterns and trends.

Loss Function​

A loss function in machine learning is a method of evaluating how well the algorithm models the given data. If predictions deviate from actual results, the loss function outputs a higher number; the goal is to minimize this loss.

Example: In a regression model, a common loss function is mean squared error, which measures the average of the squares of the errors between predicted and actual values.

Quantization​

In the context of AI, quantization involves reducing the precision of the numbers used to represent a model's parameters, which can reduce the model size and speed up inference, often with minimal impact on accuracy.

Example: Converting a neural network's weights from 32-bit floating-point to 8-bit integers to deploy the model on mobile devices with limited memory.

Adjacent Terms​

AR (Augmented Reality)​

Augmented Reality (AR) is a technology that superimposes computer-generated enhancements, like images or data, atop an existing reality, enhancing the user's perception of the real world.

Example: Using AR glasses that display directions and information about nearby landmarks as you walk through a city.

VR (Virtual Reality)​

Virtual Reality (VR) is an immersive technology that creates a simulated environment, often completely different from the real world, where users can interact with 3D worlds using special headsets or gloves.

Example: A VR gaming setup where players enter a fully immersive, computer-generated landscape, experiencing it as if they were physically present.

API (Application Programming Interface)​

An API is a set of rules and protocols for building and interacting with software applications. It defines the way different software programs and components should interact with each other.

Example: A weather application on your phone using an API to fetch data from a remote weather server.

Quantum Computing​

Quantum Computing is an area of computing focused on developing computer technology based on the principles of quantum theory, which explains the nature and behavior of energy and matter on the quantum (atomic and subatomic) level. Quantum computers use quantum bits or qubits, which can represent and store more complex information more efficiently than classical bits.

Example: A quantum computer solving complex computational problems in fields like cryptography, which are impractical for classical computers.

Edge Computing​

Edge Computing refers to computing processes that are conducted at or near the source of data generation, rather than relying on a central data-processing warehouse. This approach reduces latency and bandwidth use.

Example: Processing data from IoT devices locally on a smart home device instead of sending the data to a distant cloud server for processing.

IoT (Internet of Things)​

The Internet of Things (IOT) is a network of physical devices embedded with sensors, software, and other technologies for the purpose of connecting and exchanging data with other devices and systems over the internet.

Example: Smart thermostats in homes that learn the owner's schedule and preferences, adjusting the home's temperature automatically.

Cloud Computing​

Cloud Computing is the delivery of different services through the internet, including data storage, servers, databases, networking, and software. Cloud-based storage makes data accessible from anywhere with an internet connection.

Example: Using an online service like Google Drive to store personal files, which can be accessed from any device with internet access.