NewsScraper

A personalized news aggregator that scrapes RSS feeds from multiple sources, filters articles using AI based on your interests, and compiles relevant news with summaries and images.

Features

• AI-Powered Filtering: Uses Groq's Llama 3.3 model to intelligently filter articles based on your custom keywords
• Multi-Source Scraping: Aggregates news from:
  • Sports (AS.com - Football & Tennis)
  • Spanish Politics (20minutos)
  • US Politics (NPR)
  • Technology & AI (Ars Technica, Wired)
• Image Extraction: Automatically extracts relevant images from RSS feeds
• Smart Deduplication: Filters out tracking pixels and low-quality images
• Secure Configuration: API keys stored safely in environment variables

Prerequisites

• Python 3.12 or higher
• Groq API key (free at console.groq.com)

Installation

1. Clone the repository

   git clone https://github.com/yourusername/NewsScraper.git
   cd NewsScraper

2. Install dependencies

   pip install feedparser requests groq python-dotenv

3. Set up environment variables

   Create a .env file in the project root:

   GROQ_API_KEY=your_groq_api_key_here

   Get your free Groq API key from: https://console.groq.com/keys

Usage

Run the news scraper:

python NewsScraper.py

This will:

1. Fetch articles from all configured RSS feeds
2. Filter them using AI based on your keywords
3. Extract summaries and images
4. Display organized results by category

Configuration

Customize News Sources

Edit the RSS_FEEDS dictionary in NewsScraper.py:

RSS_FEEDS = {
    "deportes": [
        "https://feeds.as.com/mrss-s/pages/as/site/as.com/section/futbol/portada/",
    ],
    "politica_es": [
        "https://www.20minutos.es/rss/"
    ],
    # Add your own RSS feeds here
}

Customize Keywords

Edit the KEYWORDS dictionary to filter articles based on your interests:

KEYWORDS = {
    "deportes": ["real madrid", "alcaraz", "carlos alcaraz"],
    "politica": ["elecciones", "gobierno", "trump", "Ukraine"],
    "tech": ["ia", "python", "startups", "machine learning"]
}

Project Structure

NewsScraper/
├── NewsScraper.py       # Main script
├── .env                 # Environment variables (not committed)
├── .gitignore          # Git ignore file
├── README.md           # This file
└── test_*.py           # Test files (optional)

How It Works

1. RSS Feed Parsing: Uses feedparser to fetch articles from RSS feeds
2. AI Filtering: Sends article titles to Groq's Llama 3.3 model with custom prompts
3. Image Extraction:
   • Different strategies for different sources (AS.com, NPR, Wired, etc.)
   • Filters out tracking pixels and invalid images
4. Data Aggregation: Collects summaries and images for relevant articles

Example Output

🤖 Iniciando News Scraper...
============================================================

📰 Obteniendo noticias de DEPORTES...
[['Real Madrid wins La Liga...', 'https://image-url.jpg'], ...]

🇪🇸 Obteniendo noticias de POLÍTICA ESPAÑOLA...
[['Spanish elections update...', 'https://image-url.jpg'], ...]

🇺🇸 Obteniendo noticias de POLÍTICA USA...
[['Epstein files released...', 'https://image-url.jpg'], ...]

💻 Obteniendo noticias de TECH/AI...
[['AI agents breakthrough...', 'https://image-url.jpg'], ...]

============================================================
✅ Scraping completado!

Security

• API keys are stored in .env file (never committed to git)
• .gitignore prevents sensitive files from being pushed
• Environment variables loaded using python-dotenv

Future Enhancements

• Web dashboard interface
• Database storage for historical data
• Sentiment analysis
• Article deduplication across sources

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is open source and available under the MIT License.

Author

Jaime - GitHub Profile

Acknowledgments

• RSS feeds provided by AS.com, 20minutos, NPR, Ars Technica, and Wired
• AI filtering powered by Groq
• Built with Python and feedparser