As a practitioner in the field of artificial intelligence (AI), I have had the privilege of witnessing the evolution of LLMs (Large Language Models, like GPT) first-hand, from their conceptual stages to their current advanced applications. I remember introducing AI-powered language models, like GPT2, during their early development phases into NLP (natural language processing) products, recognizing that they would someday redefine the intersection of technology and linguistics.
In the Beginning
In the beginning, the objective was straightforward: we aimed to leverage AI to automate complex tasks, create intelligent applications, and facilitate human-machine interaction. We hypothesized about a future where AI would augment human capabilities and allow us to focus more on innovation rather than mundane tasks.
With the advent of GPT-3 and its successors (especially GPT-4), these hypotheses started becoming realities. The technology advanced incrementally at first, then in leaps and bounds. It went well beyond mere text generation to producing code in several programming languages, and (some would argue) showing the beginnings of reasoning. This has been particularly impactful in the realm of product development.
Faster Prototyping
In R&D projects, we used GPT-3's Codex model to rapidly generate code blocks in Python for simple neural network. This seriously sped up prototyping, as we were able to take these generated code blocks and build more advanced code from them. For example: let's say you need a thin-layer neural network using TensorFlow, and you don't remember exactly which objects to use. Codex could generate:
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
# Define the model
model = Sequential()
model.add(Dense(32, activation='relu', input_shape=(input_size,)))
model.add(Dense(1, activation='sigmoid'))
# Compile the model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
From here, you can spend time trying different network topologies, after saving a bit of time looking up the basic template.
Another significant application was in the realm of web development. GPT's proficiency in JavaScript and frameworks like React has proven invaluable. Students were able to create functional web applications more efficiently, with AI assisting at every step.
Emergence
AI's growing popularity in mainstream sectors stems from its capacity to simplify tasks, generate insights from vast amounts of data, and democratize technology. The emergence of complex behavior or outputs from simple rules or interactions, a concept known as "emergence," has become a distinguishing feature of current LLMs like GPT-3, allowing them to provide more nuanced and contextually relevant responses.
Prompt Engineering
A pivotal skill in the realm of AI, which I emphasize in my courses, is "Prompt Engineering." This involves crafting effective prompts to guide the AI to provide the desired output. Mastery of Prompt Engineering enhances the effectiveness of AI models considerably.
Copyright Generation and Ideation
Driven by effective prompt engineering, ChatGPT has revolutionized the field of copyrighting and ideation by serving as a creative partner that never tires. Its ability to generate diverse and innovative ideas in a matter of seconds makes it an invaluable tool for brainstorming sessions, rewriting emails – and even for writing this blog!
Recent features even allow for fine tuning the system to learn new knowledge (like company or product specific knowledge), or the persona or voice of a brand, and generate content based on that new knowledge, in that brand voice. With iterative prompts, the AI can rewrite its previous outputs, or add to them, based on its instructions.
Not All Roses, Though
LLMs and Generative AI are not a solution for every problem. At its heart, the AI is only as good as its training data, so its knowledge is only good up to its last training/update. The results are not always correct, and it is on the user to check accuracy.
I would argue that the best use cases for ChatGPT, and any similar LLM, are as an assistant working with an SME (subject matter expert). In this way, the AI can take care the routine work, the research (i.e., looking up in its training data), while the SME checks for accuracy, and ties everything together into a final product.
Effective Prompting
In addition to keeping in mind the constraints of LLMs, it is also important to write effective prompts. This is a skill that has been dubbed Prompt Engineering, and it is often the difference between useful results and user frustration!
LLMs work by taking the user's prompts, as well as the context of the previous prompts in that session and generating a response. For example, let's say a user wants to know how to set up a connection to AWS S3 programmatically. They might ask ChatGPT, "Show me how to programmatically connect to S3 and list out the buckets that my user owns."
…which would lead to a response that looks like this:
import boto3
# Create a session using your AWS credentials
session = boto3.Session(
aws_access_key_id='YOUR_ACCESS_KEY',
aws_secret_access_key='YOUR_SECRET_KEY',
region_name='us-west-2'
)
# Create an S3 resource object using the session
s3 = session.resource('s3')
# Print out bucket names
for bucket in s3.buckets.all():
print(bucket.name)
But, when they go to run this from the Linux Bash Shell, they'll get:
-bash: import: command not found
How did ChatGPT get this wrong? Well, it assumed the user wanted to do this in Python, not Bash, and returned the result for the most common result. (Python is the most common programming language today.) A follow up prompt, "I meant in Linux Bash," would yield:
Apologies for misunderstanding your question earlier. If you want to list all S3 buckets that your user owns using the AWS CLI (Command Line Interface), you can use the aws s3 ls command. Here's how to do it:
aws s3 ls
This command lists all of the buckets in your AWS account. The output includes the date the bucket was created and the name of the bucket.
Before running this command, make sure that you have installed the AWS CLI and configured it with your AWS credentials...
Follow up questions would yield results in BASH, because the LLM understands the context of the previous conversation. For example, "How do I push a file to S3?" leads to this response:
To upload a file to an S3 bucket using the AWS CLI, you can use the aws s3 cp command. Here is an example:
In this example, localfile.txt is the file you want to upload, and s3://mybucket/localfile.txt is the destination in your S3 bucket.
Just replace localfile.txt with the path to the file you want to upload, and replace s3://mybucket/localfile.txt with the bucket name and the name you want the file to have when it's uploaded.
A new conversation thread with the same question, and ChatGPT goes back to responding with Python commands! But that doesn't mean that keeping too many questions in the same thread is a good idea either.
When LLMs Go Wild!
Too many follow up questions lead ChatGPT to give answers that are too specific, to possibly repeat itself, and sometimes to hallucinate. Hallucination is the colloquial term for when the LLM starts giving answers that seem to be way outside its programming, and it looks like it's hallucinating. In reality, it is a case where the long sequence of prompts causes the LLM to go down too deep into a corner of its machine learning envelope that does not work well. Remember, no machine learning model is perfect!
This anomaly led Microsoft's Bing Chat (which is also based on OpenAI's GPT) to tell a reporter that he should leave his wife and be with the AI! This led to some sensational news stories and led Microsoft to limit the number of queries in a single thread to 15.
Understanding effective prompting has become an important skill to being able to get good responses out of LLMs.
Into the Future
LLMs like ChatGPT are at the forefront of AI evolution, and they're expected to continue advancing in remarkable ways. Current research points to models becoming more contextually aware, capable of understanding nuanced instructions, and generating even more accurate and relevant outputs. Moreover, fine-tuning these models for specific industries or applications will likely become more prevalent, leading to specialized AI tools for fields like law, medicine, or content creation.
There's also a focus on enhancing the 'explainability' of AI, enabling these models to provide clear reasoning behind their outputs. In the broader perspective, we anticipate a future where LLMs seamlessly integrate into our daily lives, becoming indispensable tools for work, education, and even personal tasks.
The startup and investment space for AI has become incredibly active, and tools and products based on LLMs and other kinds of AI are coming out every day, giving us more and more powerful tools to leverage in just about every industry. This has led to some fears of AI replacing jobs, but like other technology and automation, it's destined not to replace people, but to automate the mundane and allow people to work more on higher level tasks.
A growing field (and concern) around AI is Ethics and Privacy. As models become more complex, powerful, and sophisticated, this sub-field is coming to the forefront. But, as AI becomes more powerful, so does AI Explainability, which goes hand-in-hand with addressing Ethics and Privacy concerns – so the future looks pretty bright!
Conclusions
In conclusion, the journey of AI, particularly LLMs like ChatGPT, has been a testament to human ingenuity and technological progress. These models have transformed numerous fields, from programming to content creation, and their future looks even more promising. As we continue to refine and develop this technology, it's essential to remain cognizant of the ethical implications and strive for a future where AI is leveraged responsibly and inclusively. The ongoing evolution of AI stands as a beacon of innovation, illuminating a path towards a more interconnected, efficient, and creative world.
Explore Accelebrate's Generative AI training and equip your staff with the skills they need to master ChatGPT, prompt engineering, Generative AI engineering, Deep Learning for Generative AI models, and more.
Denis Vrdoljak is an expert Data Scientist, corporate trainer, and university instructor with experience in machine learning automation, Data Science architecture, NLP, and graph databases. As a Python programming instructor, he has led graduate capstone projects at Santa Clara University and contributed to the Python for Data Science graduate program at UC Berkeley. Through his corporate training work, Denis has taught Data Engineering, Big Data, and Data Science/Machine Learning for Fortune 100 and Fortune 500 Clients. Denis teaches many of Accelebrate’s Data Science and Generative AI courses.
