fss-mini-rag-github/agent-user-testing/01-mechanical-engineering/INSTRUCTIONS.md

# Test Scenario 01: Mechanical Engineering - CAD Standards Intelligence

## 🔧 **Industry Context**: Mechanical Engineering Firm
**Role**: Junior Mechanical Engineer  
**Task**: Build a smart CAD standards knowledge base that can instantly answer design questions

## 📋 **Scenario Description**
You're a junior mechanical engineer at an automotive parts manufacturer. Your team constantly struggles to find specific CAD modeling standards, tolerance requirements, and design guidelines buried in hundreds of pages of documentation. You'll use FSS-Mini-RAG to create an intelligent system that can instantly answer any CAD standards question.

## 🎯 **Your Mission (Completely Autonomous)**

### **Step 1: Setup FSS-Mini-RAG**
1. Read the repository README.md to understand how to install FSS-Mini-RAG
2. Follow the installation instructions for your platform
3. Verify the installation works by running `rag-mini --help`

### **Step 2: Gather CAD Standards Documentation**
Create a folder called `cad-standards-docs` and populate it with relevant documentation:
- Download ASME Y14.5 geometric dimensioning and tolerancing standards
- Get ISO 2768 general tolerance standards documentation  
- Find automotive CAD modeling best practices guides
- Include SolidWorks/AutoCAD design guidelines
- Add manufacturing DFM (Design for Manufacturing) standards

**Pro tip**: Look for PDF standards documents, technical guides, and industry best practices.

### **Step 3: Build Your Intelligent Knowledge Base**
```bash
# Navigate to your research folder
cd cad-standards-docs

# Initialize FSS-Mini-RAG index  
rag-mini init

# Check the index was created successfully
rag-mini stats

# Get system info to verify everything is working
rag-mini info
```

### **Step 4: Test Your CAD Standards Oracle**
Now for the cool part - your documentation is now searchable with natural language! Test these engineering queries:

```bash
# Search for tolerance information
rag-mini search "What are the standard tolerances for holes in automotive suspension components?"

# Find CAD modeling guidelines  
rag-mini search "How should CAD assembly models be structured for manufacturing?"

# Query design standards
rag-mini search "What geometric tolerancing symbols are required for shaft fits?"

# Search for file organization
rag-mini search "What are the file naming conventions for engineering drawings?"

# Look for quality requirements
rag-mini search "What inspection requirements exist for automotive safety components?"
```

### **Step 5: Advanced Engineering Searches**
Try these more sophisticated queries:

```bash
# Search for specific functions (if you have code documentation)
rag-mini find-function "tolerance_calculation" 

# Look for classes in programming guides
rag-mini find-class "DrawingTemplate"

# Update your knowledge base (when you add new documents)
rag-mini update

# Get detailed statistics about your knowledge base
rag-mini stats
```

### **Step 6: Document Your Engineering Intelligence System**
Write your findings in `RESULTS.md` including:

#### **Knowledge Base Performance**
- How many documents were indexed?
- How fast were the search responses?
- Which types of questions worked best?

#### **Real Engineering Value** 
- Can you quickly find specific tolerance requirements?
- Does it help locate CAD best practices efficiently?
- How does this compare to manual PDF searching?

#### **Professional Impact**
- How much time would this save during design reviews?
- Could this help with compliance and standards verification?
- What would be the value for training new engineers?

### **Step 7: Complete Professional Evaluation**
Rate FSS-Mini-RAG's effectiveness for:
- **Finding specific engineering standards** (1-10)
- **Answering tolerance and design questions** (1-10)  
- **Helping with CAD workflow optimization** (1-10)
- **Overall usefulness for mechanical engineering** (1-10)

### **Step 8: Document Your Experience**
Create a comprehensive `RESULTS.md` including:

#### **Executive Summary**
- What you built (CAD Standards Intelligence System)
- Key findings and success metrics
- Professional impact assessment

#### **Technical Details**
- Number of documents indexed and file sizes
- Search response times and accuracy ratings
- Most effective query types and examples
- Command usage statistics (init, search, stats, etc.)

#### **Professional Value Assessment**
- Time saved compared to manual document searching
- Potential impact on design review processes
- Training value for new engineers
- Compliance and standards verification improvements

#### **User Experience Report**
- Installation process evaluation
- Command usability ratings
- Documentation quality assessment
- Suggested improvements or missing features

### **Step 9: Repository Contribution Workflow**

#### **Repository Information**
- **Repository URL**: `http://192.168.1.3:3000/foxadmin/fss-mini-rag-github.git`
- **Main Branch**: `main`
- **Testing Branch**: `agent-user-testing` (where scenarios are located)

#### **Branch Management**
```bash
# Clone the repository
git clone http://192.168.1.3:3000/foxadmin/fss-mini-rag-github.git
cd fss-mini-rag-github

# Start from the agent-user-testing branch
git checkout agent-user-testing

# Create your own branch for your results
git checkout -b agent-test-mechanical-engineering-$(date +%Y%m%d)

# Navigate to your scenario
cd agent-user-testing/01-mechanical-engineering/
```

#### **Submit Your Results**
```bash
# Add your completed RESULTS.md
git add RESULTS.md

# Commit with descriptive message
git commit -m "Agent Test Results: Mechanical Engineering CAD Standards Intelligence

- Tested FSS-Mini-RAG with automotive CAD standards documentation
- Created intelligent knowledge base for tolerance and design queries
- Evaluated semantic search effectiveness for engineering workflows
- Documented professional impact and time-saving potential
- Rating: [X]/10 overall effectiveness"

# Push your branch
git push origin agent-test-mechanical-engineering-$(date +%Y%m%d)
```

#### **Create Pull Request**
```bash
# Use gitea CLI to create PR
gitea prs create "Agent Test: Mechanical Engineering Results" agent-test-mechanical-engineering-$(date +%Y%m%d) agent-user-testing --body "Completed comprehensive testing of FSS-Mini-RAG for mechanical engineering workflows.

## Test Summary
- Built CAD Standards Intelligence System
- Indexed [X] engineering documents
- Tested [X] search queries with [X]% accuracy
- Overall effectiveness rating: [X]/10

## Key Findings
[Brief summary of major discoveries]

## Professional Impact
[Assessment of real-world value for engineers]

## Recommendations
[Suggestions for improvements or additional features]"
```

### **Step 10: Validation Requirements**

Your submission must include:

#### **Required Evidence**
- ✅ **Screenshots** of successful `rag-mini init` and `rag-mini stats` output
- ✅ **Search examples** with actual query results (at least 5 different searches)
- ✅ **Performance metrics** (response times, index size, document count)
- ✅ **Professional assessment** with specific use cases and value propositions

#### **Quality Standards**
- ✅ **Functional completeness**: All major commands tested (init, search, stats, info)
- ✅ **Real-world relevance**: Actual industry documents and realistic queries
- ✅ **Professional writing**: Clear, actionable insights for engineering teams
- ✅ **Quantitative data**: Specific metrics and measurable outcomes

#### **Submission Checklist**
- [ ] Created intelligent knowledge base successfully
- [ ] Tested minimum 5 different search queries
- [ ] Documented all command usage and results
- [ ] Provided professional impact assessment
- [ ] Created proper git branch with descriptive name
- [ ] Submitted PR with comprehensive description
- [ ] Included evidence screenshots/outputs
- [ ] Met all validation requirements

## 📁 **Final Deliverables**
- `cad-standards-docs/` folder with indexed technical documentation
- `RESULTS.md` with comprehensive evaluation and evidence
- Git branch with proper commit history
- Pull request with detailed description
- Professional assessment of FSS-Mini-RAG effectiveness

## ⏱️ **Expected Duration**: 3-4 hours (including documentation and PR submission)

## 🎉 **Success Outcome**
You'll have created an **intelligent CAD standards assistant** AND provided valuable feedback to improve FSS-Mini-RAG for engineering professionals!

## 🎓 **Learning Objectives**
- Experience semantic search with technical engineering content
- Evaluate AI-powered documentation assistance for professional workflows  
- Test real-world applicability of RAG systems in mechanical engineering
- Practice professional software evaluation and contribution workflows