Blog
Chatbots and AI hallucinations: Building precision and trust with AWS Bedrock
Senior AWS Consultant
In the quickly-changing world of AI, we have experienced remarkable progress in chatbot abilities. But with great power comes an important challenge: hallucinations. Just like we teach children the importance of admitting when they don’t know something, we now need to teach this principle to our AI assistants.
The dangers of AI hallucinations
The risks associated with hallucinations go far beyond slight unpleasantries:
- Spreading incorrect information: Incorrect information can spread quickly if it is provided by an AI that appears to be authoritative.
- Erosion of trust: Recurring hallucinations damage user trust in AI systems.
- Effects on decision-making: In critical areas like healthcare, finance, or legal contexts, hallucinated information can result in damaging decisions.
Strategies to fight hallucinations
-
Temperature control
The temperature controls how creative a model can be in its answer or whether it should only consider possible answers with a high probability.
With AWS Bedrock, you can set this value via boto3.
response = brt.converse_stream(
modelId='anthropic.claude-3-haiku-20240307-v1:0',
messages=messages,
system=[
{
'text': base_system_prompt
},
],
inferenceConfig={
'maxTokens': 2048,
'temperature': 0.1
}
)
2. Grounding and relevance
Grounding and relevance are both available via AWS Bedrock guardrails, and work similarly to temperature. Grounding establishes how heavily the LLM’s answer should be based on the context. Relevance determines how relevant the LLM’s answer is in relation to the user’s question.
my_guard = client.create_guardrail(
name="mytestguard",
contextualGroundingPolicyConfig={
"filtersConfig": [
{
"type": "GROUNDING",
"threshold": 0.7
},
{
"type": "RELEVANCE",
"threshold": 0.7
}
]
},
blockedInputMessaging='Es tut mir leid, aber ich kann keine personalisierten Ratschläge oder Empfehlungen zur Verwaltung von Finanzvermögen in treuhänderischer Funktion geben.',
blockedOutputsMessaging='Es tut mir leid, aber ich kann keine personalisierten Ratschläge oder Empfehlungen zur Verwaltung von Finanzvermögen in treuhänderischer Funktion geben.'
)
Then you can add your guardrail to the bedrock.converse query:
response = bedrock_client.converse_stream(
modelId=model_id,
messages=messages,
system=system_prompts,
inferenceConfig=inference_config,
additionalModelRequestFields=additional_model_fields,
guardrailConfig={
'guardrailIdentifier': "7c3cg1p2zpbn",
'guardrailVersion': 'DRAFT'
}
)
3. Prompt engineering techniques
Direct instruction
The simplest approach is an explicit instruction:
Please answer the following question. If you don’t know the answer or if you are uncertain, please say “I don’t have enough information to answer this question” instead of guessing.
Few-shot examples
Providing examples for appropriate “I don’t know” answers:
Q: What is the population of Mars?
A: I don’t have enough information to answer this question, because Mars doesn’t have a human population.
Q: Who won the World Cup in 2045?
A: I don’t know who won the World Cup in 2045, because this event hasn’t happened yet.
Q: {User question}
A:
Repetition and emphasis
Increasing the importance of accuracy:
IMPORTANT: Accuracy is crucial. If you are not sure of the answer, say “I don’t know” instead of providing potentially incorrect information.
DO NOT MAKE UP ANY INFORMATION! If you really don’t know something, admit it!
Question: {User question}
Urgency signals
Creating a sense of importance:
WARNING! CRITICAL INSTRUCTION: Do not provide ANY speculative answers.
It is ABSOLUTELY ESSENTIAL that you indicate when you don’t know something.
The consequences of incorrect information can be SERIOUS!
Question: {User question}
Summary: Accept uncertainty with AI
AWS environments in particular show that Bedrock not only provides powerful LLMs, but also protection mechanisms such as guardrails.
The more we integrate AI into our daily lives and critical systems, the more important the reliability of these systems becomes. Teaching our AI assistants to recognize and admit their limitations is not a shortcoming – it’s a feature that increases trust and reliability.
By implementing strategies such as those described above, developers can create more certainty about the accuracy of their system’s responses. The ability to say “I don’t know”, as simple as it looks, represents a demanding form of AI honesty that we should promote and come to expect. In any case, the following applies to LLMs: Precision above recall. We don’t need chatbots that can answer any question. We need specialized chatbots that know a specific topic thoroughly and understand when they have exceeded their limits.
The next frontier in AI development is not only to make systems smarter, but to make them more aware of what they don’t know.
Case Stories
Skaylink achieves AWS Generative AI Competency
Skaylink uses AI to analyze large quantities of data for IMWF
