GitHub - IndexFlowing/IndexFlow-core: Open-source search engine indexing infrastructure with inline SEO quality gates, fair multi-site scheduling, and rolling quota circuit breaker.

🚀 What is IndexFlow? IndexFlow is an open-source search engine indexing infrastructure built with Rust. Modern websites need more than good content and keywords. Search engines must first be able to discover, crawl, and understand the pages that matter. IndexFlow provides lightweight, developer-friendly Rust libraries and infrastructure for managing this technical layer. It focuses on: Search engine discovery Sitemap processing Technical SEO validation URL indexing workflows Search engine submission Multi-site indexing infrastructure The goal is simple: Make search engine visibility easier to build, monitor, and manage. 🎯 Why IndexFlow? SEO often starts with keywords, rankings, and backlinks. But before any of that, there is a more fundamental question: Can search engines reliably access and understand your website? IndexFlow focuses on this technical foundation. 🔍 Search Discovery Search engines need structured information to discover website content efficiently. IndexFlow provides components for working with: XML Sitemaps Sitemap indexes URL structures Search indexing workflows ⚙ Technical SEO Many indexing problems originate from basic technical issues. IndexFlow can help identify issues such as: Missing canonical tags Incorrect robots directives Invalid metadata Technical HTML problems Other SEO quality issues 🌐 Search Visibility Infrastructure IndexFlow is designed to provide the infrastructure layer between websites and search engines. Instead of building every indexing-related capability from scratch, developers can use reusable components and build higher-level workflows on top of them. 🏗 Architecture IndexFlow is designed as a reusable Rust ecosystem. IndexFlow │ ┌──────────────┴──────────────┐ │ │ ▼ ▼ indexflow-sitemap indexflow-seo XML Sitemap Parser Technical SEO Analyzer │ │ └──────────────┬──────────────┘ │ ▼ IndexFlow Platform The open-source libraries provide the foundation. The future IndexFlow Platform will build additional capabilities on top of these components, including indexing workflows, monitoring, and multi-site management. 📦 Open Source Components indexflow-sitemap A lightweight Rust library for parsing and processing XML sitemaps. Features XML sitemap parsing Sitemap index support URL extraction Developer-friendly API Crates.io: indexflow-sitemap Documentation: docs.rs indexflow-seo A Rust library for technical SEO analysis and website quality checks. Features HTML analysis SEO metadata checking Technical SEO validation Extensible analyzer architecture Crates.io: indexflow-seo Documentation: docs.rs ⚡ Quick Start Add the components you need to your Rust project. For example: [dependencies] indexflow-sitemap = "..." Or: [dependencies] indexflow-seo = "..." Then build your own SEO and indexing workflows using the libraries provided by IndexFlow. See the individual crate documentation for detailed usage examples. 💡 Design Philosophy 🦀 Rust Native IndexFlow is built with Rust to provide: Memory safety High performance Reliable infrastructure components Zero-cost abstractions 👨💻 Developer First IndexFlow is designed for developers who want: Open-source solutions Self-hosted options Transparent architecture Reusable libraries ⚡ Lightweight Infrastructure Instead of building another large and complex SEO stack, IndexFlow focuses on: Simple deployment Clear architecture Composable components Reusable infrastructure 🛣 Roadmap ✅ Available indexflow-sitemap XML sitemap parser Sitemap index support URL extraction Published on crates.io indexflow-seo Technical SEO analysis SEO quality checks Extensible analyzer architecture Published on crates.io 🚧 Building IndexFlow Platform Planned capabilities include: Website management Search indexing workflows Sitemap monitoring Google Search Console integration IndexNow integration Multi-site management URL scheduling and quota management 🔮 Future Search & AI Visibility Intelligence Exploring: AI-generated SEO reports GEO optimization insights Website intelligence Search visibility analysis 🌐 IndexFlow Platform The open-source components provide the technical foundation for the future IndexFlow Platform. The platform is intended to help website owners and developers manage: Website health Search visibility Indexing workflows Multiple websites Search engine integrations Learn more: www.indexflowing.com 🤝 Contributing IndexFlow is built in the open. Contributions, discussions, bug reports, and feedback are welcome. If you find IndexFlow useful, consider giving the repository a ⭐ Star. It helps the project reach more developers and contributors. 📄 License Licensed under either: MIT License Apache License 2.0 at your option.