Skip to content
/ PyLoPi Public

🧩 PyLoPi is an intelligent, modular log monitoring system that uses AI to detect, analyze, and resolve errors across multiple platforms with real-time dashboards and automated solutions

License

Notifications You must be signed in to change notification settings

PicoBaz/PyLoPi

Repository files navigation

🔍 PyLoPi - Python Log Processor Intelligence

PyLoPi Logo Python Flask License

An intelligent, modular log monitoring and analysis system with AI-powered error detection and automated solutions

FeaturesInstallationUsageArchitectureConfigurationContributing


🌟 Features

Core Functionality

  • 🤖 Intelligent Error Detection: Automatically detects 15+ error types across multiple programming languages and platforms
  • 🔍 AI-Powered Analysis: Provides detailed analysis of errors with context-aware explanations
  • 💡 Automated Solutions: Suggests fixes and generates code snippets to resolve issues
  • 📧 Email Notifications: Real-time alerts with direct links to error analysis
  • 🌐 Web Crawler Integration: Searches for solutions across the web
  • 📊 Real-time Dashboard: Live monitoring with beautiful visualizations

Technical Highlights

  • 🧩 Modular Architecture: Plugin-based system for easy extensibility
  • 🌍 Multilingual Support: Built-in English and Persian (Farsi) localization
  • ⚡ High Performance: Efficient log tailing with minimal resource usage
  • 💾 SQLite Database: Lightweight, zero-configuration data storage
  • 🎨 Modern UI: Responsive design with gradient animations and glass morphism
  • 🔒 Secure: No external dependencies for core functionality

Supported Error Types

✓ SyntaxError          ✓ TypeError           ✓ ValueError
✓ AttributeError       ✓ NameError           ✓ ImportError
✓ IndexError           ✓ KeyError            ✓ FileNotFoundError
✓ PermissionError      ✓ RuntimeError        ✓ MemoryError
✓ RecursionError       ✓ ZeroDivisionError   ✓ HTTP Errors (404, 500, 403)
✓ DatabaseError        ✓ ConnectionError     ✓ TimeoutError

🚀 Installation

Prerequisites

  • Python 3.8 or higher
  • pip (Python package manager)
  • Modern web browser (Chrome, Firefox, Safari, Edge)

Quick Start

  1. Clone the repository
git clone https://github.com/PicoBaz/PyLoPi.git
cd PyLoPi
  1. Create virtual environment (recommended)
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
  1. Install dependencies
pip install -r requirements.txt
  1. Run the application
python app.py
  1. Open your browser
http://localhost:5000

📖 Usage

Basic Usage

  1. Start the application
python app.py
  1. Configure log paths

    • Navigate to the Dashboard
    • Add log file paths you want to monitor
    • Click "Start Monitoring"
  2. View analysis

    • Errors appear in real-time on the dashboard
    • Click "View Details" for full analysis
    • Get automated solutions and code fixes

Advanced Configuration

Email Notifications

  1. Go to Settings tab
  2. Enable email notifications
  3. Configure SMTP settings:
{
    "email_notifications": true,
    "email_address": "[email protected]",
    "smtp_server": "smtp.gmail.com",
    "smtp_port": 587,
    "smtp_username": "[email protected]",
    "smtp_password": "your-app-password"
}

Error Type Filtering

Select which error types to monitor:

  • Navigate to Settings
  • Check/uncheck error types
  • Click "Save Settings"

Supported Log Formats

PyLoPi works with various log formats:

Python Applications

import logging
logging.basicConfig(filename='app.log', level=logging.ERROR)

Web Servers (Apache/Nginx)

ErrorLog /var/log/apache2/error.log

Custom Applications

  • Any text-based log file
  • Supports UTF-8 encoding
  • Real-time tailing

🏗️ Architecture

Project Structure

pylopi/
├── app.py                 # Main Flask application
├── database.py            # Database manager
├── log_analyzer.py        # Core analysis engine
├── email_notifier.py      # Email notification system
├── config_manager.py      # Configuration handler
├── requirements.txt       # Python dependencies
├── config.json           # User configuration (auto-generated)
├── pylopi.db             # SQLite database (auto-generated)
├── templates/
│   └── index.html        # Web interface
└── README.md             # This file

System Flow

graph LR
    A[Log Files] --> B[Log Analyzer]
    B --> C[Error Detection]
    C --> D[Web Crawler]
    D --> E[Solution Generation]
    E --> F[Database]
    F --> G[Web Dashboard]
    E --> H[Email Notifier]
Loading

Component Details

1. Log Analyzer (log_analyzer.py)

  • Real-time log monitoring
  • Pattern matching for error detection
  • Context-aware analysis
  • Solution generation

2. Database Manager (database.py)

  • SQLite-based storage
  • Thread-safe operations
  • Efficient querying
  • Statistics generation

3. Email Notifier (email_notifier.py)

  • HTML email templates
  • SMTP integration
  • Severity-based alerting
  • Direct dashboard links

4. Config Manager (config_manager.py)

  • JSON-based configuration
  • Default settings
  • Hot-reload support
  • Validation

⚙️ Configuration

Configuration File (config.json)

{
    "language": "en",
    "email_notifications": false,
    "email_address": "",
    "smtp_server": "smtp.gmail.com",
    "smtp_port": 587,
    "smtp_username": "",
    "smtp_password": "",
    "enabled_error_types": [
        "SyntaxError",
        "TypeError",
        "ValueError",
        "AttributeError",
        "NameError",
        "ImportError",
        "IndexError",
        "KeyError",
        "FileNotFoundError",
        "404Error",
        "500Error",
        "DatabaseError",
        "ConnectionError"
    ],
    "log_retention_days": 30,
    "max_logs_per_file": 10000,
    "monitoring_interval": 2
}

Environment Variables

You can also use environment variables:

export PYLOPI_SMTP_USERNAME="[email protected]"
export PYLOPI_SMTP_PASSWORD="your-app-password"
export PYLOPI_PORT=5000

🔌 Extending PyLoPi

Adding New Error Types

Edit log_analyzer.py:

self.error_patterns = {
    'CustomError': r'CustomError:(.+)',
    # Add your pattern here
}

Creating Custom Analyzers

Create a new file custom_analyzer.py:

class CustomAnalyzer:
    def analyze(self, log_line):
        # Your custom logic
        return analysis_result

Adding New Languages

Edit the translations object in templates/index.html:

const translations = {
    es: {
        title: "PyLoPi",
        subtitle: "Monitor y Análisis Inteligente de Logs",
        // Add more translations
    }
};

📊 API Endpoints

REST API

Endpoint Method Description
/api/config GET Get current configuration
/api/config POST Update configuration
/api/start-monitoring POST Start log monitoring
/api/stop-monitoring POST Stop log monitoring
/api/logs GET Get recent logs
/api/log/<id> GET Get log details
/api/stats GET Get statistics
/api/language POST Set interface language

Example API Usage

import requests

response = requests.post('http://localhost:5000/api/start-monitoring', 
    json={'log_paths': ['/var/log/app.log']})

logs = requests.get('http://localhost:5000/api/logs?limit=10').json()

🎨 UI Features

Dashboard

  • Live Statistics: Total logs, today's count, critical errors
  • Recent Logs: Latest errors with quick preview
  • Monitoring Control: Start/stop monitoring interface
  • Log Path Management: Add/remove paths dynamically

Logs View

  • Filterable List: All logged errors
  • Severity Indicators: Color-coded badges
  • Quick Access: Direct links to full analysis

Log Detail View

  • Full Analysis: Complete error breakdown
  • Solution Steps: Step-by-step fix instructions
  • Code Snippets: Ready-to-use code fixes
  • Full Log Context: Original log entry

Settings

  • Email Configuration: SMTP setup
  • Error Filters: Choose which errors to track
  • Language Selection: Switch between English/Persian
  • Preferences: Customize monitoring behavior

🧪 Testing

Manual Testing

# Create a test log file
echo "TypeError: unsupported operand type(s)" >> test.log

# Start PyLoPi and add test.log
# Error should appear in dashboard

Unit Tests (Coming Soon)

python -m pytest tests/

🛠️ Troubleshooting

Common Issues

Port already in use

# Change port in app.py
app.run(debug=True, host='0.0.0.0', port=5001)

Email notifications not working

  • Enable "Less secure app access" for Gmail
  • Use app-specific password for Gmail
  • Check SMTP settings

Logs not appearing

  • Verify file path is correct
  • Check file permissions
  • Ensure log file exists

Database locked error

  • Close other connections to pylopi.db
  • Restart the application

🤝 Contributing

We welcome contributions! Here's how:

  1. Fork the repository
  2. Create a feature branch
    git checkout -b feature/amazing-feature
  3. Commit your changes
    git commit -m 'Add amazing feature'
  4. Push to the branch
    git push origin feature/amazing-feature
  5. Open a Pull Request

Contribution Guidelines

  • Follow PEP 8 style guide
  • Add docstrings to functions
  • Update README for new features
  • Test thoroughly before submitting

👥 Authors


🙏 Acknowledgments

  • Flask framework for the web interface
  • BeautifulSoup for web crawling
  • SQLite for reliable data storage
  • The open-source community

📞 Support


Made with ❤️ by developers, for developers

About

🧩 PyLoPi is an intelligent, modular log monitoring system that uses AI to detect, analyze, and resolve errors across multiple platforms with real-time dashboards and automated solutions

Resources

License

Contributing

Stars

Watchers

Forks

Packages

No packages published

Contributors 2

  •  
  •