6.3.2. Custom Question Answering

Unlike CLU which extracts intents for triggering actions, Custom Question Answering returns answer text directly to users. Use it when you need to surface information from FAQs, documentation, or policy content.

💡 First Principle: FAQ-style questions have known answers that don't change—"What are your return policy hours?" always has the same answer. Question Answering handles this by matching user questions (however phrased) to a static knowledge base of Q&A pairs. The key distinction from RAG: QA is for static, curated answers (FAQs, policies); RAG is for dynamic, document-based answers (searching across changing content). The exam tests this distinction frequently.

🔧 Implementation Reference: Custom Question Answering

Knowledge Base Sources:

Testable Pattern:

from azure.ai.language.questionanswering import QuestionAnsweringClient
from azure.core.credentials import AzureKeyCredential

client = QuestionAnsweringClient(endpoint, AzureKeyCredential(key))

response = client.get_answers(
    question="What is your return policy?",
    project_name="RetailFAQ",
    deployment_name="production",
    confidence_threshold=0.5,  # Minimum confidence to return answer
    top=3  # Number of answers to return
)

for answer in response.answers:
    print(f"Answer: {answer.answer}")
    print(f"Confidence: {answer.confidence:.2%}")
    print(f"Source: {answer.source}")

Multi-Turn Conversations:

Multi-turn enables follow-up questions that maintain context from the previous Q&A exchange. Configure follow-up prompts in Language Studio to guide users through complex topics.

# Multi-turn with context from previous answer
response = client.get_answers(
    question="How long do I have?",  # Follow-up question
    project_name="RetailFAQ",
    deployment_name="production",
    answer_context=AnswerSpanRequest(
        previous_question="What is your return policy?",
        previous_answer_id=previous_answer.id
    )
)

Active Learning:

Active learning improves your knowledge base by learning from user interactions. When multiple answers have similar confidence scores, the system can suggest alternatives and learn from user selections.

Synonyms and Alterations:

Synonyms help match user queries that use different terminology than your knowledge base content. Add synonyms in Language Studio under the "Synonyms" tab.

Error Handling Pattern:

from azure.ai.language.questionanswering import QuestionAnsweringClient
from azure.core.exceptions import HttpResponseError, ResourceNotFoundError

try:
    response = client.get_answers(
        question=user_question,
        project_name="RetailFAQ",
        deployment_name="production",
        confidence_threshold=0.5
    )
    
    if not response.answers or response.answers[0].confidence < 0.5:
        # No confident answer found
        return "I don't have information about that. Would you like to speak with a human?"
        
except ResourceNotFoundError:
    logging.error("Knowledge base project or deployment not found")
except HttpResponseError as e:
    if e.status_code == 400:
        logging.error("Invalid request format")
    elif e.status_code == 429:
        time.sleep(int(e.response.headers.get("Retry-After", 60)))

CLI Equivalent (REST):

curl -X POST "https://{endpoint}/language/:query-knowledgebases?projectName=RetailFAQ&deploymentName=production&api-version=2023-04-01" \
  -H "Ocp-Apim-Subscription-Key: {key}" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "What is your return policy?",
    "top": 3,
    "confidenceScoreThreshold": 0.5
  }'

⚠️ Exam Trap: Synonyms are for query matching, not alternate question text. To add alternative phrasings for a question, add them as separate questions mapped to the same answer.

⚠️ Exam Trap: Confidence threshold filters results—if no answers meet the threshold, an empty result is returned, not the best available answer.