Atlas Knowledge Base

A decentralized knowledge base graph network for collaborative information sharing and discovery.

Links

A decentralized knowledge base graph network for collaborative information sharing and discovery.

Overview

Atlas Knowledge Base lets you create decentralized "knowledge vaults" that live on the Internet Computer. You can upload markdown, JSON, media and archives, tag everything semantically, and explore the relationships as an interactive graph. The public dApp focuses on a smooth, visual way to browse and discover connections in your knowledge, while the Architecture & Roadmap section below dives into how it all works under the hood.

To Do

A cartography of Wikipedia pages around data visualization
2d/3d node viewer in tkd (sigma.js/D3)

Key Features

Decentralized knowledge graph storage
Semantic linking and relationship mapping
Collaborative knowledge curation
Token-based content validation
Cross-referencing and citation tracking
AI-powered knowledge discovery

Technology Stack

Backend: Motoko canisters on the Internet Computer, managed with Mops
Frontend: Vite + SvelteKit + TypeScript
Identity: Internet Identity for user authentication
Tooling: mo-dev, ESLint, Prettier, pic.js for IC canister testing

Development Status

Status: Active development (built on the Vite + SvelteKit + Motoko starter template)

Architecture & Roadmap

Atlas Knowledge Base – Decentralized Semantic Knowledge Engine
Project Status: Active Development
Technology Stack: SvelteKit, Internet Computer Protocol (ICP), CanDB, Motoko
Repository: ~/repos/atlas-knowledge-base
Template Base: Vite + SvelteKit + Motoko starter

Overview

Atlas Knowledge Base is a scalable semantic knowledge engine built on decentralized cloud architecture. It provides a sophisticated platform for organizing, visualizing, and exploring interconnected knowledge through graph-based structures, semantic tagging, and advanced file management capabilities.

Core Capabilities

Knowledge Organization

Semantic Tagging System: Hierarchical tag structures with parent-child relationships
Multi-format File Support: Markdown, JSON, ZIP archives, audio files, and general media
Metadata Extraction: Automatic frontmatter parsing from markdown files
Categorized Storage: Structured organization using vault-based partition keys

Graph Visualization

Interactive Knowledge Graphs: Dynamic visualization using Sigma.js and Graphology
Force-Directed Layouts: ForceAtlas2 algorithm for natural graph clustering
Node Relationships: Visual representation of file-tag connections and hierarchies
Search and Filtering: Real-time graph filtering by categories and search terms
Expandable Clusters: Interactive exploration of grouped knowledge nodes

File Processing Engine

Chunk-Based Storage: Large file handling through 2MB chunk splitting
Batch Upload Processing: ZIP file extraction with automatic metadata mapping
Markdown Processing: Frontmatter tag extraction and content rendering
Progress Tracking: Real-time upload progress with toast notifications
Duplicate Detection: SHA256-based file identification and deduplication

Technical Architecture

Backend Infrastructure

Multi-Canister Design: Separate services for Atlas, App, User, and Index management
CanDB Integration: Distributed database with automatic scaling
Motoko Smart Contracts: Type-safe backend with comprehensive error handling
Identity Management: Internet Identity integration for secure authentication

Data Model Structure

		vault#<atlas-slug>/
metadata                    # Atlas metadata and configuration
file#<filename>            # File metadata and content references
tag#<tag>#<file>          # Tag-to-file relationship mapping
chunk#<file>#<id>         # File content chunks for large files

In addition to the vault records above, the Atlas schema also defines:

An app partition (PK = app, SK = app) for global application metadata such as visit counters and versioning
A user partition (PK = user, SK = principal#<principal>) for per-identity data tied to Internet Identity principals
Together with the vault#<vault> keys, this gives Atlas a compact, single-table style layout on top of CanDB.

Frontend Features

Responsive Design: Mobile-first interface with dark/light mode support
Command Menu: Keyboard-driven navigation and search (K support)
Data Tables: Sortable, filterable views with checkbox selection
Modal System: Drawer-based UI components for mobile optimization
Real-time Updates: Live progress tracking and status notifications

Advanced Functionality

Semantic Capabilities

Hierarchical Tagging: Support for nested tag structures (e.g., books/philosophy/ethics)
Cross-Reference Discovery: Automatic relationship detection between related content
Metadata Enrichment: Frontmatter parsing with custom attribute support
Content Indexing: Full-text search across file contents and metadata

Batch Operations

ZIP Archive Processing: Automatic extraction and processing of compressed knowledge bases
Metadata Inheritance: JSON sidecar files for enhanced file metadata
Bulk Tag Management: Array-based tag imports and updates
Content Validation: File type checking and format validation

Graph Analytics

Node Clustering: Automatic grouping of related knowledge components
Edge Weight Calculation: Relationship strength visualization
Path Discovery: Finding connections between disparate knowledge nodes
Centrality Analysis: Identifying key knowledge hubs and connectors

File Format Support

Primary Formats

Markdown (.md): Full frontmatter support with tag extraction
JSON (.json): Tag arrays and metadata objects
ZIP Archives: Batch processing with metadata mapping
Audio Files: Media metadata extraction and storage

Processing Pipeline

Upload Detection: File type identification and routing
Metadata Extraction: Frontmatter/JSON parsing for tags and attributes
Content Chunking: Large file splitting for blockchain storage
Relationship Mapping: Tag-to-file association creation
Graph Integration: Node and edge generation for visualization

Knowledge Graph Features

Visualization Engine

Sigma.js Integration: High-performance graph rendering
Custom Layouts: ForceAtlas2 physics simulation for natural clustering
Interactive Navigation: Pan, zoom, and click-to-explore functionality
Category Filtering: Dynamic node visibility based on type classification

Node Types

File Nodes: Represent uploaded content with metadata
Tag Nodes: Hierarchical keyword categorization
Cluster Nodes: Aggregated groups of related content
Relationship Edges: Weighted connections between entities

Development Features

Testing Infrastructure

Playwright E2E Tests: Comprehensive browser automation testing
Motoko Unit Tests: Backend logic validation using Mops
PicJS Integration: Canister testing framework for IC development
Continuous Integration: Automated test pipelines

Developer Experience

Live Reload: mo-dev integration for backend development
Type Safety: Full TypeScript coverage with generated Candid bindings
Code Formatting: Prettier with Motoko and Svelte plugins
Component Library: Reusable UI components with shadcn/ui styling

Scalability Architecture

Storage Scaling

Partition-Based Design: Automatic canister scaling as data grows
Chunk Distribution: Load balancing across multiple storage canisters
Index Optimization: Efficient querying through structured sort keys
Cache Management: Strategic caching for frequently accessed content

Performance Optimization

Lazy Loading: On-demand content fetching for large datasets
Progress Streaming: Real-time feedback during bulk operations
Parallel Processing: Concurrent file uploads and processing
Memory Efficiency: Streaming operations for large file handling

Atlas Knowledge Base represents a sophisticated approach to decentralized knowledge management, combining semantic organization with powerful visualization tools and robust file processing capabilities, all built on censorship-resistant blockchain infrastructure.