From eb35a4269c6dd63aa32d015431fe9fd383b4231e Mon Sep 17 00:00:00 2001 From: FSSCoding Date: Sat, 25 Oct 2025 16:41:12 +1100 Subject: [PATCH] Add credentials management system for 3 accounts per provider type Credentials Directory Structure: - credentials/gmail/ - Gmail OAuth credentials (3 accounts) - credentials/outlook/ - Outlook/Microsoft365 OAuth credentials (3 accounts) - credentials/imap/ - IMAP username/password credentials (3 accounts) Files Added: - credentials/README.md - Comprehensive setup guide - credentials/*/account1.json.example - Templates for each provider Security: - Updated .gitignore to exclude actual credential files - Only .example files are tracked in git - README includes security best practices Setup Instructions: - Gmail: OAuth 2.0 via Google Cloud Console - Outlook: OAuth 2.0 via Azure Portal with Microsoft Graph API - IMAP: Username/password (supports Gmail app passwords) Dependencies Verified: - Gmail: google-api-python-client, google-auth-oauthlib (installed) - Outlook: msal, requests (installed) - IMAP: Python standard library (no additional deps) Usage: - --credentials credentials/gmail/account1.json - --credentials credentials/outlook/account2.json - --credentials credentials/imap/account3.json All providers now support 3 accounts each with organized credential storage. --- .gitignore | 3 +- credentials/README.md | 261 ++++++++++++++++++++++ credentials/gmail/account1.json.example | 11 + credentials/imap/account1.json.example | 7 + credentials/outlook/account1.json.example | 6 + 5 files changed, 287 insertions(+), 1 deletion(-) create mode 100644 credentials/README.md create mode 100644 credentials/gmail/account1.json.example create mode 100644 credentials/imap/account1.json.example create mode 100644 credentials/outlook/account1.json.example diff --git a/.gitignore b/.gitignore index 6225fad..7497f42 100644 --- a/.gitignore +++ b/.gitignore @@ -21,7 +21,8 @@ maildir # Credentials .env -credentials/ +credentials/**/*.json +!credentials/**/*.json.example *.json !config/*.json !config/*.yaml diff --git a/credentials/README.md b/credentials/README.md new file mode 100644 index 0000000..4817794 --- /dev/null +++ b/credentials/README.md @@ -0,0 +1,261 @@ +# Email Sorter - Credentials Management + +This directory stores authentication credentials for email providers. The system supports up to 3 accounts of each type (Gmail, Outlook, IMAP). + +## Directory Structure + +``` +credentials/ +├── gmail/ +│ ├── account1.json # Primary Gmail account +│ ├── account2.json # Secondary Gmail account +│ ├── account3.json # Tertiary Gmail account +│ └── account1.json.example # Template +├── outlook/ +│ ├── account1.json # Primary Outlook account +│ ├── account2.json # Secondary Outlook account +│ ├── account3.json # Tertiary Outlook account +│ └── account1.json.example # Template +└── imap/ + ├── account1.json # Primary IMAP account + ├── account2.json # Secondary IMAP account + ├── account3.json # Tertiary IMAP account + └── account1.json.example # Template +``` + +## Gmail Setup + +### 1. Create OAuth Credentials + +1. Go to [Google Cloud Console](https://console.cloud.google.com) +2. Create a new project (or select existing) +3. Enable Gmail API +4. Go to "Credentials" → "Create Credentials" → "OAuth client ID" +5. Choose "Desktop app" as application type +6. Download the JSON file +7. Save as `credentials/gmail/account1.json` (or account2.json, account3.json) + +### 2. Credential File Format + +```json +{ + "installed": { + "client_id": "YOUR_CLIENT_ID.apps.googleusercontent.com", + "project_id": "your-project-id", + "auth_uri": "https://accounts.google.com/o/oauth2/auth", + "token_uri": "https://oauth2.googleapis.com/token", + "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs", + "client_secret": "YOUR_CLIENT_SECRET", + "redirect_uris": ["http://localhost"] + } +} +``` + +### 3. Usage + +```bash +# Account 1 +python -m src.cli run --source gmail --credentials credentials/gmail/account1.json --limit 1000 + +# Account 2 +python -m src.cli run --source gmail --credentials credentials/gmail/account2.json --limit 1000 + +# Account 3 +python -m src.cli run --source gmail --credentials credentials/gmail/account3.json --limit 1000 +``` + +## Outlook Setup + +### 1. Register Azure AD Application + +1. Go to [Azure Portal](https://portal.azure.com/#blade/Microsoft_AAD_RegisteredApps) +2. Click "New registration" +3. Name your app (e.g., "Email Sorter") +4. Choose "Accounts in any organizational directory and personal Microsoft accounts" +5. Set Redirect URI to "Public client/native" with `http://localhost:8080` +6. Click "Register" +7. Copy the "Application (client) ID" +8. (Optional) Create a client secret in "Certificates & secrets" for server apps + +### 2. Configure API Permissions + +1. Go to "API permissions" +2. Click "Add a permission" +3. Choose "Microsoft Graph" +4. Select "Delegated permissions" +5. Add: + - Mail.Read + - Mail.ReadWrite +6. Click "Grant admin consent" (if you have admin rights) + +### 3. Credential File Format + +```json +{ + "client_id": "YOUR_AZURE_APP_CLIENT_ID", + "client_secret": "YOUR_CLIENT_SECRET_OPTIONAL", + "tenant_id": "common", + "redirect_uri": "http://localhost:8080" +} +``` + +**Note:** `client_secret` is optional for desktop apps using device flow authentication. + +### 4. Usage + +```bash +# Account 1 +python -m src.cli run --source outlook --credentials credentials/outlook/account1.json --limit 1000 + +# Account 2 +python -m src.cli run --source outlook --credentials credentials/outlook/account2.json --limit 1000 + +# Account 3 +python -m src.cli run --source outlook --credentials credentials/outlook/account3.json --limit 1000 +``` + +## IMAP Setup + +### 1. Get IMAP Credentials + +For Gmail IMAP: +1. Enable 2-factor authentication on your Google account +2. Go to https://myaccount.google.com/apppasswords +3. Generate an "App Password" for "Mail" +4. Use this app password (not your real password) + +For Outlook/Office365 IMAP: +- Host: `outlook.office365.com` +- Port: `993` +- Use your regular password or app password + +### 2. Credential File Format + +```json +{ + "host": "imap.gmail.com", + "port": 993, + "username": "your.email@gmail.com", + "password": "your_app_password_or_password", + "use_ssl": true +} +``` + +### 3. Usage + +```bash +# Account 1 +python -m src.cli run --source imap --credentials credentials/imap/account1.json --limit 1000 + +# Account 2 +python -m src.cli run --source imap --credentials credentials/imap/account2.json --limit 1000 + +# Account 3 +python -m src.cli run --source imap --credentials credentials/imap/account3.json --limit 1000 +``` + +## Security Notes + +### Important Security Practices + +1. **Never commit credentials to git** + - The `.gitignore` file excludes `credentials/` directory + - Only `.example` files should be committed + +2. **File permissions** + - Set restrictive permissions: `chmod 600 credentials/*/*.json` + +3. **Credential rotation** + - Rotate credentials periodically + - Revoke unused credentials in provider dashboards + +4. **Separation** + - Keep each account's credentials in separate files + - Use descriptive names (account1, account2, account3) + +### Credential Storage Locations + +**This directory** (`credentials/`) is for: +- Development and testing +- Personal use +- Single-user deployments + +**NOT recommended for:** +- Production servers (use environment variables or secret managers) +- Multi-user systems (use proper authentication systems) +- Public repositories (credentials would be exposed) + +## Troubleshooting + +### Gmail Issues + +**Error: "credentials_path required"** +- Ensure you're passing `--credentials` flag +- Verify file exists and path is correct + +**Error: "GMAIL DEPENDENCIES MISSING"** +- Install dependencies: `pip install google-api-python-client google-auth-oauthlib` + +**Error: "CREDENTIALS FILE NOT FOUND"** +- Check file exists at specified path +- Ensure filename is correct (case-sensitive) + +### Outlook Issues + +**Error: "client_id required"** +- Verify JSON file has `client_id` field +- Check Azure app registration + +**Error: "OUTLOOK DEPENDENCIES MISSING"** +- Install dependencies: `pip install msal requests` + +**Authentication timeout** +- Complete device flow authentication within time limit +- Check browser for authentication prompt +- Verify Azure app has correct permissions + +### IMAP Issues + +**Error: "Authentication failed"** +- For Gmail: Use app password, not regular password +- Enable "Less secure app access" if using regular password +- Verify username/password are correct + +**Connection timeout** +- Check host and port are correct +- Verify firewall isn't blocking IMAP port +- Test connection with: `telnet imap.gmail.com 993` + +## Testing Credentials + +Test each credential file before running full classification: + +```bash +# Test Gmail connection +python -m src.cli test-gmail --credentials credentials/gmail/account1.json + +# Test Outlook connection +python -m src.cli test-outlook --credentials credentials/outlook/account1.json + +# Test IMAP connection +python -m src.cli test-imap --credentials credentials/imap/account1.json +``` + +## Dependencies + +### Gmail +```bash +pip install google-api-python-client google-auth-oauthlib google-auth-httplib2 +``` + +### Outlook +```bash +pip install msal requests +``` + +### IMAP +No additional dependencies required (uses Python standard library). + +--- + +**Remember:** Keep your credentials secure and never share them publicly! diff --git a/credentials/gmail/account1.json.example b/credentials/gmail/account1.json.example new file mode 100644 index 0000000..fef4e34 --- /dev/null +++ b/credentials/gmail/account1.json.example @@ -0,0 +1,11 @@ +{ + "installed": { + "client_id": "YOUR_CLIENT_ID.apps.googleusercontent.com", + "project_id": "your-project-id", + "auth_uri": "https://accounts.google.com/o/oauth2/auth", + "token_uri": "https://oauth2.googleapis.com/token", + "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs", + "client_secret": "YOUR_CLIENT_SECRET", + "redirect_uris": ["http://localhost"] + } +} diff --git a/credentials/imap/account1.json.example b/credentials/imap/account1.json.example new file mode 100644 index 0000000..c142219 --- /dev/null +++ b/credentials/imap/account1.json.example @@ -0,0 +1,7 @@ +{ + "host": "imap.gmail.com", + "port": 993, + "username": "your.email@gmail.com", + "password": "your_app_password_or_password", + "use_ssl": true +} diff --git a/credentials/outlook/account1.json.example b/credentials/outlook/account1.json.example new file mode 100644 index 0000000..b2a4195 --- /dev/null +++ b/credentials/outlook/account1.json.example @@ -0,0 +1,6 @@ +{ + "client_id": "YOUR_AZURE_APP_CLIENT_ID", + "client_secret": "YOUR_CLIENT_SECRET_OPTIONAL", + "tenant_id": "common", + "redirect_uri": "http://localhost:8080" +}