What is Artificial Intelligence?
"AI is intelligence exhibited by machines, where a machine can learn from information (data) and then use that learned knowledge to do something." Julie Mehan in Artificial Intelligence: Ethical, Social, and Security Impacts for the Present and Future
AI is usually contrasted to human intelligence. When you consider the variety in the kinds of intelligence a human brain can exhibit, the breadth of computing tasks that can be considered AI quickly becomes apparent.
Artificial intelligence : ethical, social, and security impacts for the present and the future by
This book explores the complex topic of AI ethics in a cross-functional way, alternating between theory and practice. Practical and recent events, and their associated challenges, are presented, such as the collateral effects of the COVID-19 pandemic on the application of AI technologies.
ISBN: 9781787783713Publication Date: 2022
What is Machine Learning?
"Machine learning is a subset of artificial intelligence. This field of computer science explores a computer’s ability to learn from data." Andrew Lowe and Steve Lawless in Artificial Intelligence Foundations: Learning From Experience.
Machine learning involves developing algorithms that allow computers to improve their performance at specified tasks.
Artificial Intelligence Foundations by
This introductory guide provides the understanding you need to start building artificial intelligence (AI) capability into your organization. You will learn how AI is being utilized today and how it is likely to be used in the future to balance the talents of humans and machines. You will explore robotics and machine learning within the context of AI, and discover how the challenges AI presents are being addressed.
ISBN: 1780175302Publication Date: 2021-02-19
What is a large language model?
"Large language models (LLMs) are AI models that are designed to understand and generate human language, code, and much more." In this context, "understand" means accurately interpreting and identifying context, not full human understanding. Sinan Ozdemir in The Quick Start Guide to Large Language Models
Quick Start Guide to Large Language Models by
The advancement of Large Language Models (LLMs) has revolutionized the field of Natural Language Processing in recent years. Models like BERT, T5, and ChatGPT have demonstrated unprecedented performance on a wide range of NLP tasks, from text classification to machine translation. Despite their impressive performance, the use of LLMs remains challenging for many practitioners. This book provides an overview of the key concepts and techniques used in LLMs and explains how these models work and how they can be used for various NLP tasks. The book also covers advanced topics, such as fine-tuning, alignment, and information retrieval while providing practical tips and tricks for training and optimizing LLMs for specific NLP tasks.
ISBN: 9780138199197Publication Date: 2023-10-26
What is ChatGPT?
A chatbot that uses a large language model and machine learning to successfully mimic human conversational patterns. You can think of it like the predictive text on a smart phone - it is trained to answer questions in a way that sounds plausible. It has not been trained to answer questions accurately.
Learn more about how ChatGPT works:
Is ChatGPT a Search Engine?
No. If you ask ChatGPT for lists of sources, it will provide a list, but it is guessing at which words seem to be plausible responses to the question posed rather than consulting any kind of bibliography or authoritative list of sources that actually exist. It can often use its knowledge of which words frequently appear near each other to successfully identify plausible publications, researchers, or authors for a topic, but you should always double check any citations provided by ChatGPT with an external source before assuming they are accurate. Do not simply ask ChatGPT if the citations are real - it will not provide a useful response. Sometimes it will even argue with you when you point out that a citation it provided is not real.
If you're not sure if a citation is real, please ask a librarian. Let us know that ChatGPT provided the citation.
Fake ChatGPT Citations in the News:
- The ChatGPT Lawyer Explains himself (New York Times)In a cringe-inducing court hearing, a lawyer who relied on A.I. to craft a motion full of made-up case law said he “did not comprehend” that the chat bot could lead him astray.
- ChatGPT is making up fake Guardian articles. Here’s how we’re responding. (The Guardian)The risks inherent in the technology, plus the speed of its take-up, demonstrate why it’s so vital that we keep track of it.
- We Asked ChatGPT About Art Theory. It Led Us Down a Rabbit Hole So Perplexing We Had to Ask Hal Foster for a Reality Check (Art Net)A simple ChatGPT inquiry turned into a very strange ride.
What is a Generative Search Engine?
While ChatGPT is not a search engine, there is a new kind of "generative search engine" that does intend to provide accurate and verifiable responses to natural language questions. Sometimes they succeed. Most of these tools provide both a brief answer to a question and supporting links where users can verify the information provided in the response.
In general, these tools do a much better job than ChatGPT at linking to sources they used to answer questions, but you should always click through to read the cited pieces in full. Do not assume the summary responses provided by the chat tools are accurate. Generative Search Engines cannot always identify an article that answers the question that was asked, nor can they always accurately summarize the information provided in their citations. Additionally, they have a strong bias towards popular sources. If you specifically request a scholarly source, they may make up citation or link to a popular website.
Learn more about the accuracy of Generative Search Engines:
- Evaluating Verifiability in Generative Search Engines by Nelson F. Liu, Tianyi Zhang, and Percy LiangGenerative search engines directly generate responses to user queries, along with in-line citations. A prerequisite trait of a trustworthy generative search engine is verifiability, i.e., systems should cite comprehensively (high citation recall; all statements are fully supported by citations) and accurately (high citation precision; every cite supports its associated statement). We conduct human evaluation to audit four popular generative search engines -- Bing Chat, NeevaAI, Perplexity.ai, and YouChat -- across a diverse set of queries from a variety of sources (e.g., historical Google user queries, dynamically-collected open-ended questions on Reddit, etc.). We find that responses from existing generative search engines are fluent and appear informative, but frequently contain unsupported statements and inaccurate citations: on average, a mere 51.5% of generated sentences are fully supported by citations and only 74.5% of citations support their associated sentence. We believe that these results are concerningly low for systems that may serve as a primary tool for information-seeking users, especially given their facade of trustworthiness. We hope that our results further motivate the development of trustworthy generative search engines and help researchers and users better understand the shortcomings of existing commercial systems.
Explore Generative Search Engines:
What is an AI Research Assistant?
AI Research Assistants take natural language questions or "seed papers" and return citations for scholarly papers and books (e.g. Elicit, Research Rabbit, etc.). This class of tools is designed for professional researchers seeking peer-reviewed scholarship, so they tend to do better at retrieving genuine citations for scholarly work than either ChatGPT or Generative Search Engines. However, researchers using the tools should be mindful of limitations in the indexes they search - none is as comprehensive as Google Scholar or the Summon Index, nor do they tend to have the quality control of a tool like Web of Science or a disciplinary index. Typically there is very little information available on the algorithms used to rank results. Some attempt to summarize the findings of an article, others simply display the traditional abstract written by humans. Library staff recommend not assuming that AI summaries are accurate.
As of August 2023 the Oberlin Library staff have not yet found a scientific study evaluating the effectiveness of these tools in interpreting research questions, retrieving results relevant to those questions, or in summarizing the conclusions of articles. Preliminary tests by library staff suggest that some tools may be a helpful way to kickstart a research process as long as users are mindful of potential limitations. A handful of systematic reviews and meta-analyses available in PubMed mention using elicit.org as one of the search strategies.
Learn more about AI research assistants:
- Review: ElicitQualitative product review of the Elicit AI research assistant from the Journal of the Canadian Health Library Association.
Explore AI Research Assistants:
Does ChatGPT change over time?
Yes! The model is always trying to improve itself, and changes that may make it better at one task may make it worse at another. There is relatively little transparency into which changes are being implemented at any given time and what their implications are, which is why it is important to double-check information provided by it with another source. Double-checking is important even when you've heard from a reliable evaluator that it is usually good at a particular task.
Learn more about how ChatGPT's performance at specific tasks changes over time:
- How Is ChatGPT’s Behavior Changing over Time? by Lingjiao Chen, Matei Zaharia, James ZouGPT-3.5 and GPT-4 are the two most widely used large language model (LLM) services. However, when and how these models are updated over time is opaque. Here, we evaluate the March 2023 and June 2023 versions of GPT-3.5 and GPT-4 on four diverse tasks: 1) solving math problems, 2) answering sensitive/dangerous questions, 3) generating code and 4) visual reasoning. We find that the performance and behavior of both GPT-3.5 and GPT-4 can vary greatly over time. For example, GPT-4 (March 2023) was very good at identifying prime numbers (accuracy 97.6%) but GPT-4 (June 2023) was very poor on these same questions (accuracy 2.4%). Interestingly GPT-3.5 (June 2023) was much better than GPT-3.5 (March 2023) in this task. GPT-4 was less willing to answer sensitive questions in June than in March, and both GPT-4 and GPT-3.5 had more formatting mistakes in code generation in June than in March. Overall, our findings shows that the behavior of the same LLM service can change substantially in a relatively short amount of time, highlighting the need for continuous monitoring of LLM quality.
How can I tell if text was generated by ChatGPT or AI?
This is very hard to do, and you should be wary of people and tools who claim to be able to do this successfully! Asking the tools that generated the text, like ChatGPT, will not work - they are simply not trained to identify their own output and don't perform that task accurately, but they can provide seemingly-authoritative responses anyway.
As far as external evaluations go, Weber-Wulf et. al. tested 14 AI detection tools and found that they struggled to identify text created by AI if the text was slightly rewritten by a human or even a machine paraphrasing tool. Jackesch, Hancock, and Namaan found that humans struggle to identify AI generated text even in training scenarios where they are provided immediate feedback on incorrect selections and receive financial incentives for correct identifications.
Learn more about accuracy in AI-detection:
- Human heuristics for AI-generated language are flawed by Maurice Jackesch, Jeffrey Hancock, and Mor NaamanHuman communication is increasingly intermixed with language generated by AI. Across chat, email, and social media, AI systems suggest words, complete sentences, or produce entire conversations. AI-generated language is often not identified as such but presented as language written by humans, raising concerns about novel forms of deception and manipulation. Here, we study how humans discern whether verbal self-presentations, one of the most personal and consequential forms of language, were generated by AI. In six experiments, participants (N = 4,600) were unable to detect self-presentations generated by state-of-the-art AI language models in professional, hospitality, and dating contexts. A computational analysis of language features shows that human judgments of AI-generated language are hindered by intuitive but flawed heuristics such as associating first-person pronouns, use of contractions, or family topics with human-written language. We experimentally demonstrate that these heuristics make human judgment of AI-generated language predictable and manipulable, allowing AI systems to produce text perceived as “more human than human.” We discuss solutions, such as AI accents, to reduce the deceptive potential of language generated by AI, limiting the subversion of human intuition.
- AI Detection Tools are Easy to Fool by Rhiannon WilliamsMIT Technology review write up of research by Weber-Wulff, et al, evaluating the performance of AI Detection tools.
- Professor Flunks All His Students After ChatGPT Falsely Claims It Wrote Their PapersTexas A&M University–Commerce seniors who have already graduated were denied their diplomas because of an instructor who incorrectly used AI software to detect cheating.