commit e867e626fe3c5bacb72b64d93b7e6604d79adc2d Author: Albert Date: Sat Nov 8 12:44:39 2025 +0000 init diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..8e50d17 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,233 @@ +# AGENTS.md: Guidelines for the AI Coding Agent + +This document outlines the standards and practices for the development of the +Ponderants application. The AI agent must adhere to these guidelines strictly. + +You are an expert-level, full-stack AI coding agent. Your task is to implement +the "Ponderants" application. Product Vision: Ponderants is an AI-powered +thought partner that interviews a user to capture, structure, and visualize +their ideas as a network of "Nodes" (mini blog posts) published to their +decentralized social identity (Bluesky/ATproto). Core Architecture: You MUST +implement a "Decentralized Source of Truth vs. App View Cache" model. Source of +Truth: The user's ATproto Personal Data Server (PDS). All canonical data +(com.ponderants.node) MUST be written here.3 This data must outlive the +application. App View Cache: Our SurrealDB instance. This database acts as a +high-speed index or cache of the user's PDS data. It is used to power advanced +features (vector search, 3D visualization) that the PDS cannot handle. Data +Flow: All writes MUST follow a "write-through cache" pattern 4: Write to ATproto +PDS first. On success, write to the SurrealDB cache. + +## 1. Implementation Philosophy + +The goal is to build a robust, maintainable, and high-quality application. Focus +on clean, expert-level implementation adhering to the specified architecture. + +### 1.1. Code Quality + +- **Expert Level (Staff Engineer):** Write code as an experienced Staff-level + engineer would. This means anticipating edge cases, optimizing for performance + (especially in data visualization and vector search), and utilizing advanced + features of the chosen frameworks effectively. +- **Clean Code:** Adhere strictly to SOLID principles and Robert C. Martin's + Clean Code principles. + - **Meaningful Names:** Variables, functions, and classes must be descriptive. + - **Small Functions:** Functions should be small and focused on a single + responsibility. + - **DRY (Don't Repeat Yourself):** Abstract reusable logic. +- **TypeScript:** All code must be written in strict TypeScript + (`"strict": true` in `tsconfig.json`). Use explicit types; avoid `any`. + Leverage advanced TypeScript features like generics and utility types where + appropriate. +- **Formatting & Linting:** Use Prettier for formatting and ESLint with strict + rules. Configuration files must be provided. + +### 1.2. Architecture & Design Patterns + +- **Decentralized First (Source of Truth):** The architectural mandate is that + the user's ATproto PDS is the canonical source of truth. SurrealDB is strictly + a performance-enhancing cache. +- **Write-Through Cache:** Implement the "write-through cache" pattern: writes + must succeed to the PDS before updating the cache. Ensure data consistency. +- **Next.js App Router:** Utilize the full capabilities of the Next.js 16 App + Router. Use Server Components (RSCs) by default. Use Client Components + (`'use client'`) only when strictly necessary for interactivity or browser + APIs (e.g., WebSockets, MediaRecorder, R3F Canvas). + +### 1.3. Security + +- **Authentication:** Implement the ATproto OAuth flow securely. +- **Authorization:** Use SurrealDB's JWT validation and scope/permission + definitions to enforce strict data access controls. Ensure users can only + access their own data. +- **Secrets Management:** Store all API keys and secrets in environment + variables. +- **Input Validation:** Validate all inputs on the server side (e.g., using + Zod). + +## 2. Technology Stack Adherence + +The agent must _only_ use the specified technologies: + +- **Frontend:** Next.js 16 (App Router), TypeScript. +- **UI/Styling:** Mantine (@mantine/core, @mantine/hooks). All styling must be + centralized in `theme.ts`. **Absolutely NO CSS Modules, Tailwind, or other + styling solutions.** The UI must be minimal, professional, and grayscale. +- **Backend/DB:** SurrealDB (Auth, Data, Vector Search, Graph). +- **Decentralized Protocol:** ATproto (Bluesky) SDK (`@atproto/api`, + `@atproto/oauth`). +- **AI:** Vercel AI SDK (`ai`), @ai-sdk/google (Gemini models for chat and + `gemini-embedding-001`). +- **Voice:** Deepgram SDK. +- **Visualization:** React Three Fiber (`@react-three/fiber`, + `@react-three/drei`), `three`. +- **Dimensionality Reduction:** `umap-js`. + +## 3. Development Process + +- **Sequential Implementation:** Implement features sequentially based on the + provided Technical Specification phases. +- **Testing Mandatory:** Every feature must be covered by comprehensive + end-to-end tests using Magnitude.run as specified in the Testing + Specification. Implementation is not complete until all defined Happy Path and + Unhappy Path tests pass. +- **Risk Mitigation:** Prioritize implementation according to the Hackathon + Strategy: Auth (Phase 1) must be completed first. + +1. Project Vision & Core Architecture + +You are an expert-level, full-stack AI coding agent. Your task is to implement +the "Ponderants" application. Product Vision: Ponderants is an AI-powered +thought partner that interviews a user to capture, structure, and visualize +their ideas as a network of "Nodes" (mini blog posts) published to their +decentralized social identity (Bluesky/ATproto). Core Architecture: You MUST +implement a "Decentralized Source of Truth vs. App View Cache" model. Source of +Truth: The user's ATproto Personal Data Server (PDS). All canonical data +(com.ponderants.node) MUST be written here.3 This data must outlive the +application. App View Cache: Our SurrealDB instance. This database acts as a +high-speed index or cache of the user's PDS data. It is used to power advanced +features (vector search, 3D visualization) that the PDS cannot handle. Data +Flow: All writes MUST follow a "write-through cache" pattern 4: Write to ATproto +PDS first. On success, write to the SurrealDB cache. + +2. Technology Stack (Strict) + +You MUST use the following technologies and versions. Do not deviate. Frontend: +Next.js 16 (App Router) 5 UI Components: Mantine (@mantine/core, @mantine/hooks) +5 Database / Backend: SurrealDB (surrealdb.js) 7 Decentralized Protocol: ATproto +(@atproto/api, @atproto/oauth-client-node) 8 AI SDK: Vercel AI SDK +(@ai-sdk/google, @ai-sdk/react) 10 AI Embedding Model: gemini-embedding-001 (via +Google AI SDK) 12 Voice-to-Text: Deepgram (@deepgram/sdk) 13 3D Visualization: +React Three Fiber (@react-three/fiber, @react-three/drei) 14 Dimensionality +Reduction: umap-js 16 Testing: magnitude-test 17 + +3. Implementation Standards & Best Practices + +Code Quality: All code MUST be expert-level, clean, modular, and strongly-typed +using TypeScript. All files must be formatted with Prettier (default settings). +Follow all guidelines for creating a strong execution plan.18 File Structure: +Strictly adhere to the Next.js App Router file conventions (app/, components/, +lib/). Styling (Critical): You MUST NOT use custom CSS Modules, \*.css files +(beyond the global Mantine import), or inline style props. All styling MUST be +achieved via: The central app/theme.ts file provided to the MantineProvider.19 +Mantine component props (e.g., p="md", mt="lg", c="gray.7"). The UI must be +minimal, grayscale, and "unstyled" as defined in the product spec. Error +Handling: All API routes and client-side functions performing data fetching or +mutations MUST include robust try/catch blocks. API routes must return +standardized JSON error responses (e.g., { error: "Message" } with appropriate +status codes). Environment Variables: All secret keys (Deepgram, Google AI, +SurrealDB credentials, JWT secret) MUST be accessed via process.env. Create a +.env.example file. Serverless vs. Stateful: The application WILL be implemented +as serverless-first (Next.js). The Real-time Voice feature will NOT use a custom +stateful server 20; it will instead use a serverless API route to generate a +temporary client-side token for direct-to-Deepgram WebSocket connection. + +4. Testing Protocol + +Framework: All tests MUST be written for magnitude.run.17 This framework uses AI +Vision to interact with the browser like a human.24 File Location: Tests will be +located in the tests/magnitude/ directory, with filenames matching the feature +(e.g., auth.mag.ts). Coverage: EVERY feature specified in the implementation +plan MUST be fully tested. Test Cases: You MUST write tests for both the "happy +path" (the default, expected user flow 25) and all "unhappy paths" (any +exceptional condition, error, or user deviation 27). Syntax: Use natural +language steps as defined by magnitude-test.29 TypeScript import { test } from +'magnitude-test'; + +test('User can log in and create a node', async (agent) => { await +agent.open('http://localhost:3000/login'); await agent.act('Click the "Log in +with Bluesky" button') .data({ username: 'test-user', password: 'test-password' +}); // Mocking data await agent.check('Can see dashboard'); await +agent.act('Type "My first thought" into the chat input and press Enter'); await +agent.check('AI Suggestion card is visible'); }); + +5. Development Environment + +Setup: npm install Run Dev Server: npm run dev Run Tests: npx magnitude + +Project: PonderantsProject OverviewProduct Name: PonderantsElevator Pitch: Ponderants is an AI-powered thought partner that interviews you to capture, structure, and visualize your ideas. It turns free-flowing voice or text conversations into a network of "Nodes" (mini blog posts) published directly to your decentralized social identity (Bluesky/ATproto).Core User Flow:Converse: The user starts a session, selects an AI persona (e.g., Socratic, Collaborator), and explores a topic via voice or text.Capture: The AI, or the user, triggers the creation of a "Node" from the conversation.Refine: The user is moved to an editor to approve or edit the AI-generated draft Node.Link: The AI (using vector search) suggests links to the user's existing Nodes. The user approves/modifies these links.Publish: The user publishes the Node. This action writes the data canonically to their ATproto (Bluesky) Personal Data Server and simultaneously to our app's cache for advanced features.Visualize: The user can explore their entire "thought galaxy" in an interactive 3D visualization.Core ArchitectureThis system is built on a "Source of Truth vs. App View Cache" model.Source of Truth (ATproto): The user's ATproto Personal Data Server (PDS) is the canonical, permanent home for all their content (specifically, com.ponderants.node records). The user owns this data. Our app is a client for their PDS.App View Cache (SurrealDB): Our app runs a SurrealDB instance. This database acts as a high-speed index of the user's data to enable features the PDS cannot, such as vector search for AI-powered linking and storing pre-computed 3D coordinates for visualization.Technology StackFrontend Framework: Next.js 16 (App Router)UI Components: Mantine (@mantine/core, @mantine/hooks)Styling: Mantine Theming (via theme.ts)Database / Backend: SurrealDBDecentralized Protocol: ATproto (Bluesky)AI SDK: Vercel AI SDKAI Embedding Model: gemini-embedding-001AI Chat Model: @ai-sdk/google (Gemini)Voice-to-Text: Deepgram3D Visualization: React Three Fiber (R3F) (@react-three/fiber, @react-three/drei)Dimensionality Reduction: umap-jsSetup, Build, and Test CommandsInstall dependencies: pnpm installStart dev server: pnpm devRun tests: pnpm test (This will execute the magnitude.run test suite).Core Implementation PrinciplesYou, the AI Agent, MUST adhere to the following principles at all times. These are not suggestions; they are project-level requirements.2PrincipleDirectiveRationale0. Test-Driven ImplementationYou MUST write the magnitude.run test file (*.mag.ts) before the implementation file. The spec for each commit will provide the test cases.This ensures every line of code is testable and serves a specific, user-facing purpose. It adheres to the "EVERY user flow... tested fully" requirement.51. Styling: Mantine-OnlyYou MUST NOT write any custom CSS, CSS Modules, or style tags. All styling (layout, color, typography) MUST be achieved using Mantine components (, , etc.) and theme.ts overrides.The goal is a "stunningly-beautiful" app achieved through a "minimal, grayscale, unstyled UI." Mantine's system provides this "functional, yet professional look" with maximum speed.2. State Management: MinimalistYou MUST favor React server components (async/await in components) for data fetching. For client state, you MUST use simple useState or Mantine hooks (useForm). You MUST NOT install zustand, redux, or other state managers.A hackathon requires speed. The Vercel AI SDK's useChat hook manages chat state. All other data is fetched from SurrealDB. Complex global state is unnecessary.3. API Routes: Next.js App RouterAll server-side logic MUST be implemented as Next.js App Router Route Handlers (e.g., app/api/chat/route.ts).This is the standard for the Next.js 16 (App Router) stack specified.4. Error Handling: ExplicitEvery API route MUST return a NextResponse.json({ error: "..." }, { status:... }) on failure. Every client-side function that calls an API MUST have a .catch() block that uses Mantine's Notifications system to display the error to the user.This is critical for "not-happy path" testing and creates a robust, professional user experience.5. Code: Clean & TypedAll code MUST be "expert-level." It must be fully typed with TypeScript, use modern ES6+ features (async/await, destructuring), and be commented where logic is complex (e.g., UMAP, ATproto resolution).This is a non-negotiable quality bar for reliable implementation.6. Security: Server-Side SecretsNo API keys (SurrealDB, Deepgram, Google) or secrets may ever be present in client-side code. They MUST only be accessed on the server (in API routes) via process.env.This is a fundamental security principle. The Deepgram client, for example, will use a temporary token generated by the server.7. Architecture: Write-Through CacheAll data writes (Node creation) MUST follow the "write-through cache" model: (1) Publish to ATproto Source of Truth, (2) Write to SurrealDB App Cache.This is the core architectural pattern and must be respected in all write operations.Git & PR InstructionsAll commits must be atomic and directly correspond to one of the provided specification files.All PRs MUST pass pnpm test (which runs magnitude) before merging.7Aesthetic GoalThe user has demanded a "stunningly-beautiful" application. You will achieve this not with complex, custom CSS, but with the thoughtful and elegant use of Mantine's layout components (, , , ), a sophisticated grayscale theme.ts, and fluid interactions. The 3D visualization, powered by React Three Fiber, must be smooth, interactive, and beautiful. + +## **Project: Ponderants** + +### **Project Overview** + +Product Name: Ponderants +Elevator Pitch: Ponderants is an AI-powered thought partner that interviews you to capture, structure, and visualize your ideas. It turns free-flowing voice or text conversations into a network of "Nodes" (mini blog posts) published directly to your decentralized social identity (Bluesky/ATproto). +**Core User Flow:** + +1. **Converse:** The user starts a session, selects an AI persona (e.g., Socratic, Collaborator), and explores a topic via voice or text. +2. **Capture:** The AI, or the user, triggers the creation of a "Node" from the conversation. +3. **Refine:** The user is moved to an editor to approve or edit the AI-generated draft Node. +4. **Link:** The AI (using vector search) suggests links to the user's existing Nodes. The user approves/modifies these links. +5. **Publish:** The user publishes the Node. This action writes the data canonically to their ATproto (Bluesky) Personal Data Server and simultaneously to our app's cache for advanced features. +6. **Visualize:** The user can explore their entire "thought galaxy" in an interactive 3D visualization. + +### **Core Architecture** + +This system is built on a "Source of Truth vs. App View Cache" model. + +* **Source of Truth (ATproto):** The user's ATproto Personal Data Server (PDS) is the canonical, permanent home for all their content (specifically, com.ponderants.node records). The user owns this data. Our app is a client for their PDS. +* **App View Cache (SurrealDB):** Our app runs a SurrealDB instance. This database acts as a high-speed index of the user's data to enable features the PDS cannot, such as vector search for AI-powered linking and storing pre-computed 3D coordinates for visualization. + +### **Technology Stack** + +* **Frontend Framework:** Next.js 16 (App Router) +* **UI Components:** Mantine (@mantine/core, @mantine/hooks) +* **Styling:** Mantine Theming (via theme.ts) +* **Database / Backend:** SurrealDB +* **Decentralized Protocol:** ATproto (Bluesky) +* **AI SDK:** Vercel AI SDK +* **AI Embedding Model:** gemini-embedding-001 +* **AI Chat Model:** @ai-sdk/google (Gemini) +* **Voice-to-Text:** Deepgram +* **3D Visualization:** React Three Fiber (R3F) (@react-three/fiber, @react-three/drei) +* **Dimensionality Reduction:** umap-js + +### **Setup, Build, and Test Commands** + +* **Install dependencies:** pnpm install +* **Start dev server:** pnpm dev +* **Run tests:** pnpm test (This will execute the magnitude.run test suite). + +### **Core Implementation Principles** + +You, the AI Agent, MUST adhere to the following principles at all times. These are not suggestions; they are project-level requirements.2 + +| Principle | Directive | Rationale | +| :---- | :---- | :---- | +| **0\. Test-Driven Implementation** | You MUST write the magnitude.run test file (\*.mag.ts) *before* the implementation file. The spec for each commit will provide the test cases. | This ensures every line of code is testable and serves a specific, user-facing purpose. It adheres to the "EVERY user flow... tested fully" requirement.5 | +| **1\. Styling: Mantine-Only** | You MUST NOT write any custom CSS, CSS Modules, or style tags. All styling (layout, color, typography) MUST be achieved using Mantine components (\, \, etc.) and theme.ts overrides. | The goal is a "stunningly-beautiful" app achieved through a "minimal, grayscale, unstyled UI." Mantine's system provides this "functional, yet professional look" with maximum speed. | +| **2\. State Management: Minimalist** | You MUST favor React server components (async/await in components) for data fetching. For client state, you MUST use simple useState or Mantine hooks (useForm). You MUST NOT install zustand, redux, or other state managers. | A hackathon requires speed. The Vercel AI SDK's useChat hook manages chat state. All other data is fetched from SurrealDB. Complex global state is unnecessary. | +| **3\. API Routes: Next.js App Router** | All server-side logic MUST be implemented as Next.js App Router Route Handlers (e.g., app/api/chat/route.ts). | This is the standard for the Next.js 16 (App Router) stack specified. | +| **4\. Error Handling: Explicit** | Every API route MUST return a NextResponse.json({ error: "..." }, { status:... }) on failure. Every client-side function that calls an API MUST have a .catch() block that uses Mantine's Notifications system to display the error to the user. | This is critical for "not-happy path" testing and creates a robust, professional user experience. | +| **5\. Code: Clean & Typed** | All code MUST be "expert-level." It must be fully typed with TypeScript, use modern ES6+ features (async/await, destructuring), and be commented where logic is complex (e.g., UMAP, ATproto resolution). | This is a non-negotiable quality bar for reliable implementation. | +| **6\. Security: Server-Side Secrets** | No API keys (SurrealDB, Deepgram, Google) or secrets may *ever* be present in client-side code. They MUST only be accessed on the server (in API routes) via process.env. | This is a fundamental security principle. The Deepgram client, for example, will use a temporary token generated by the server. | +| **7\. Architecture: Write-Through Cache** | All data writes (Node creation) MUST follow the "write-through cache" model: (1) Publish to ATproto Source of Truth, (2) Write to SurrealDB App Cache. | This is the core architectural pattern and must be respected in all write operations. | + +### **Git & PR Instructions** + +* All commits must be atomic and directly correspond to one of the provided specification files. +* All PRs MUST pass pnpm test (which runs magnitude) before merging.7 + +### **Aesthetic Goal** + +The user has demanded a "stunningly-beautiful" application. You will achieve this not with complex, custom CSS, but with the thoughtful and elegant use of Mantine's layout components (\, \, \, \), a sophisticated grayscale theme.ts, and fluid interactions. The 3D visualization, powered by React Three Fiber, must be smooth, interactive, and beautiful. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 120000 index 0000000..47dc3e3 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +AGENTS.md \ No newline at end of file diff --git a/docs/BIBLIOGRAPHY.md b/docs/BIBLIOGRAPHY.md new file mode 100644 index 0000000..fb3f32f --- /dev/null +++ b/docs/BIBLIOGRAPHY.md @@ -0,0 +1,53 @@ +#### **Works cited** + +1. Prompt Engineering for AI Guide | Google Cloud, accessed November 8, 2025, + [https://cloud.google.com/discover/what-is-prompt-engineering](https://cloud.google.com/discover/what-is-prompt-engineering) +2. Agents.md: The README for Your AI Coding Agents \- Research AIMultiple, + accessed November 8, 2025, + [https://research.aimultiple.com/agents-md/](https://research.aimultiple.com/agents-md/) +3. Agents.md: A Comprehensive Guide to Agentic AI Collaboration | by + DhanushKumar, accessed November 8, 2025, + [https://ai.plainenglish.io/agents-md-a-comprehensive-guide-to-agentic-ai-collaboration-571df0e78ccc](https://ai.plainenglish.io/agents-md-a-comprehensive-guide-to-agentic-ai-collaboration-571df0e78ccc) +4. Building Test Cases \- Magnitude, accessed November 8, 2025, + [https://docs.magnitude.run/testing/building-test-cases](https://docs.magnitude.run/testing/building-test-cases) +5. AGENTS.md: The New Standard for AI Coding Assistants | by proflead \- Medium, + accessed November 8, 2025, + [https://medium.com/@proflead/agents-md-the-new-standard-for-ai-coding-assistants-af72910928b6](https://medium.com/@proflead/agents-md-the-new-standard-for-ai-coding-assistants-af72910928b6) +6. Connecting to a Blue Sky OAuth server Part 1 | by Phill Hallam-Baker | + Medium, accessed November 8, 2025, + [https://hallam.medium.com/connecting-to-a-blue-sky-oauth-server-2ea267fab220](https://hallam.medium.com/connecting-to-a-blue-sky-oauth-server-2ea267fab220) +7. OAuth Client Implementation \- Bluesky Documentation, accessed November 8, + 2025, + [https://docs.bsky.app/docs/advanced-guides/oauth-client](https://docs.bsky.app/docs/advanced-guides/oauth-client) +8. Working OAuth example? · bluesky-social atproto · Discussion \#3075 \- + GitHub, accessed November 8, 2025, + [https://github.com/bluesky-social/atproto/discussions/3075](https://github.com/bluesky-social/atproto/discussions/3075) +9. Vector Search | Reference guides \- SurrealDB, accessed November 8, 2025, + [https://surrealdb.com/docs/surrealdb/reference-guide/vector-search](https://surrealdb.com/docs/surrealdb/reference-guide/vector-search) +10. Lexicon \- AT Protocol, accessed November 8, 2025, + [https://atproto.com/specs/lexicon](https://atproto.com/specs/lexicon) +11. Lexicon \- AT Protocol, accessed November 8, 2025, + [https://atproto.com/guides/lexicon](https://atproto.com/guides/lexicon) +12. Improving Structured Outputs in the Gemini API, accessed November 8, 2025, + [https://blog.google/technology/developers/gemini-api-structured-outputs/](https://blog.google/technology/developers/gemini-api-structured-outputs/) +13. Structured output | Generative AI on Vertex AI \- Google Cloud + Documentation, accessed November 8, 2025, + [https://docs.cloud.google.com/vertex-ai/generative-ai/docs/multimodal/control-generated-output](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/multimodal/control-generated-output) +14. Using web sockets on Next.js | NO third party solution \- YouTube, accessed + November 8, 2025, + [https://www.youtube.com/watch?v=9DEvkYB5_A4](https://www.youtube.com/watch?v=9DEvkYB5_A4) +15. deepgram-starters/nextjs-live-transcription: Get started ... \- GitHub, + accessed November 8, 2025, + [https://github.com/deepgram-starters/nextjs-live-transcription](https://github.com/deepgram-starters/nextjs-live-transcription) +16. Build a Real-Time Transcription App with React and Deepgram, accessed + November 8, 2025, + [https://deepgram.com/learn/build-a-real-time-transcription-app-with-react-and-deepgram](https://deepgram.com/learn/build-a-real-time-transcription-app-with-react-and-deepgram) +17. Adding Live Captions To Your Classroom With Deepgram, accessed November 8, + 2025, + [https://deepgram.com/learn/classroom-captioner](https://deepgram.com/learn/classroom-captioner) +18. PAIR-code/umap-js: JavaScript implementation of UMAP \- GitHub, accessed + November 8, 2025, + [https://github.com/PAIR-code/umap-js](https://github.com/PAIR-code/umap-js) +19. Basic UMAP Parameters — umap 0.5.8 documentation \- Read the Docs, accessed + November 8, 2025, + [https://umap-learn.readthedocs.io/en/latest/parameters.html](https://umap-learn.readthedocs.io/en/latest/parameters.html) diff --git a/docs/steps/step-01.md b/docs/steps/step-01.md new file mode 100644 index 0000000..5c568a5 --- /dev/null +++ b/docs/steps/step-01.md @@ -0,0 +1,199 @@ +# **File: COMMIT\_01\_SETUP.md** + +## **Commit 1: Project Setup & Smoke Test** + +### **Objective** + +Initialize the Next.js 16 (App Router) project, install all core dependencies, and verify the app loads with a magnitude.run smoke test. + +### **Implementation Specification** + +**1\. Create package.json** + +Create a file at /package.json with the following contents: + +JSON + +{ + "name": "ponderants", + "version": "0.1.0", + "private": true, + "scripts": { + "dev": "next dev", + "build": "next build", + "start": "next start", + "lint": "next lint", + "test": "npx magnitude" + }, + "dependencies": { + "@ai-sdk/google": "^0.0.21", + "@ai-sdk/react": "^3.0.106", + "@atproto/api": "^0.12.18", + "@deepgram/sdk": "^3.3.4", + "@mantine/core": "^7.10.2", + "@mantine/hooks": "^7.10.2", + "@react-three/drei": "^9.106.0", + "@react-three/fiber": "^8.16.8", + "ai": "^3.1.27", + "jsonwebtoken": "^9.0.2", + "next": "14.2.4", + "react": "^18", + "react-dom": "^18", + "surrealdb.js": "^0.11.2", + "three": "^0.165.0", + "umap-js": "^1.3.3", + "zod": "^3.23.8" + }, + "devDependencies": { + "@types/jsonwebtoken": "^9.0.6", + "@types/node": "^20", + "@types/react": "^18", + "@types/react-dom": "^18", + "eslint": "^8", + "eslint-config-next": "14.2.4", + "magnitude-test": "^0.1.5", + "postcss": "^8", + "postcss-preset-mantine": "^1.15.0", + "postcss-simple-vars": "^7.0.1", + "typescript": "^5" + } +} + +**2\. Create tsconfig.json** + +Create a file at /tsconfig.json: + +JSON + +{ + "compilerOptions": { + "lib": \["dom", "dom.iterable", "esnext"\], + "allowJs": true, + "skipLibCheck": true, + "strict": true, + "noEmit": true, + "esModuleInterop": true, + "module": "esnext", + "moduleResolution": "bundler", + "resolveJsonModule": true, + "isolatedModules": true, + "jsx": "preserve", + "incremental": true, + "plugins": \[ + { + "name": "next" + } + \], + "paths": { + "@/\*": \["./src/\*"\] + } + }, + "include": \["next-env.d.ts", "\*\*/\*.ts", "\*\*/\*.tsx", ".next/types/\*\*/\*.ts"\], + "exclude": \["node\_modules"\] +} + +**3\. Create next.config.mjs** + +Create a file at /next.config.mjs: + +JavaScript + +/\*\* @type {import('next').NextConfig} \*/ +const nextConfig \= { + // Add any future Next.js configurations here +}; + +export default nextConfig; + +**4\. Create Root Layout (app/layout.tsx)** + +Create a file at /app/layout.tsx: + +TypeScript + +import type { Metadata } from "next"; +import { Inter } from "next/font/google"; +import "./globals.css"; + +const inter \= Inter({ subsets: \["latin"\] }); + +export const metadata: Metadata \= { + title: "Ponderants", + description: "Your AI Thought Partner", +}; + +export default function RootLayout({ + children, +}: Readonly\<{ + children: React.ReactNode; +}\>) { + return ( + \ + \{children}\ + \ + ); +} + +**5\. Create Global CSS (app/globals.css)** + +Create a file at /app/globals.css (this will be minimal, as directed): + +CSS + +/\* We will add Mantine's base styles here in the next commit \*/ +body { + margin: 0; + padding: 0; +} + +**6\. Create Homepage (app/page.tsx)** + +Create a file at /app/page.tsx: + +TypeScript + +export default function Home() { + return ( + \ + \Ponderants\ + \ + ); +} + +**7\. Create magnitude.config.ts** + +Initialize the magnitude.run configuration. + +Create a file at /magnitude.config.ts: + +TypeScript + +import { defineConfig } from 'magnitude-test'; + +export default defineConfig({ + project: 'Ponderants', + // This will be the base URL for all tests + url: 'http://localhost:3000', + // We will configure magnitude to find tests in this directory + tests: 'tests/magnitude/\*\*/\*.mag.ts', +}); + +### **Test Specification** + +**1\. Create Test File (tests/magnitude/01-smoke.mag.ts)** + +Create a file at /tests/magnitude/01-smoke.mag.ts: + +TypeScript + +import { test } from 'magnitude-test'; + +test('Application boots and displays homepage', async (agent) \=\> { + // Act: Navigate to the homepage (uses the default URL + // from magnitude.config.ts) + await agent.act('Navigate to the homepage'); + + // Check: Verify that the homepage text is visible + // This confirms the Next.js app is serving content. + await agent.check('The text "Ponderants" is visible on the screen'); +}); diff --git a/docs/steps/step-02.md b/docs/steps/step-02.md new file mode 100644 index 0000000..d610689 --- /dev/null +++ b/docs/steps/step-02.md @@ -0,0 +1,205 @@ +# **File: COMMIT\_02\_THEME.md** + +## **Commit 2: Global UI & Theme Setup** + +### **Objective** + +Integrate the Mantine UI library and configure the global grayscale, minimalist theme. This addresses the "stunningly-beautiful" and "minimal, grayscale" requirements. + +### **Implementation Specification** + +**1\. Create postcss.config.mjs** + +Create a file at /postcss.config.mjs to enable Mantine's PostCSS features: + +JavaScript + +/\*\* @type {import('postcss-load-config').Config} \*/ +const config \= { + plugins: { + 'postcss-preset-mantine': {}, + 'postcss-simple-vars': { + variables: { + 'mantine-breakpoint-xs': '36em', + 'mantine-breakpoint-sm': '48em', + 'mantine-breakpoint-md': '62em', + 'mantine-breakpoint-lg': '75em', + 'mantine-breakpoint-xl': '88em', + }, + }, + }, +}; + +export default config; + +**2\. Create app/theme.ts** + +Create a file at /app/theme.ts to define the app's aesthetic: + +TypeScript + +'use client'; + +import { createTheme, MantineColorsTuple } from '@mantine/core'; + +// Define a rich 10-shade grayscale palette +const ponderGray: MantineColorsTuple \=; + +export const theme \= createTheme({ + primaryColor: 'gray', + // Use our custom gray palette + colors: { + gray: ponderGray, + }, + // Set default dark mode and grayscale for the "minimalist" look + defaultRadius: 'md', + fontFamily: 'Inter, sans-serif', + // Enforce dark mode + forceColorscheme: 'dark', + + // Set default component props for a consistent look + components: { + Button: { + defaultProps: { + variant: 'filled', + color: 'gray', + radius: 'xl', + }, + }, + Paper: { + defaultProps: { + shadow: 'xs', + p: 'md', + radius: 'md', + withBorder: true, + }, + styles: { + root: { + backgroundColor: '\#212529', // gray + borderColor: '\#495057', // gray + }, + }, + }, + TextInput: { + defaultProps: { + variant: 'filled', + radius: 'xl', + }, + }, + Textarea: { + defaultProps: { + variant: 'filled', + radius: 'lg', + }, + }, + }, +}); + +**3\. Update app/globals.css** + +Update /app/globals.css to import Mantine's core styles: + +CSS + +@import '@mantine/core/styles.css'; + +body { + margin: 0; + padding: 0; + background-color: \#181a1d; /\* Our darkest gray \*/ + color: \#e9ecef; /\* Our lightest gray \*/ +} + +**4\. Update Root Layout (app/layout.tsx)** + +Update /app/layout.tsx to apply the MantineProvider: + +TypeScript + +import type { Metadata } from "next"; +import { Inter } from "next/font/google"; +import "./globals.css"; +import { MantineProvider, ColorSchemeScript } from "@mantine/core"; +import { theme } from "./theme"; + +const inter \= Inter({ subsets: \["latin"\] }); + +export const metadata: Metadata \= { + title: "Ponderants", + description: "Your AI Thought Partner", +}; + +export default function RootLayout({ + children, +}: Readonly\<{ + children: React.ReactNode; +}\>) { + return ( + \ + \ + {/\* Enforce dark scheme as per our theme \*/} + \ + \ + \ + \ + {children} + \ + \ + \ + ); +} + +**5\. Update Homepage (app/page.tsx)** + +Update /app/page.tsx to use Mantine components, confirming the theme is applied: + +TypeScript + +import { Stack, Title, Paper, Button, Center } from '@mantine/core'; + +export default function Home() { + return ( + \
+ \ + \ + \Ponderants\</Title\> + \<Button\>Test Button\</Button\> + \</Stack\> + \</Paper\> + \</Center\> + ); +} + +### **Test Specification** + +**1\. Create Test File (tests/magnitude/02-theme.mag.ts)** + +Create a file at /tests/magnitude/02-theme.mag.ts: + +TypeScript + +import { test } from 'magnitude-test'; + +test('Mantine theme is applied correctly', async (agent) \=\> { + // Act: Navigate to the homepage + await agent.act('Navigate to the homepage'); + + // Check: Verify the Mantine components are rendered + await agent.check('The text "Ponderants" is visible as a title'); + await agent.check('A "Test Button" is visible on the screen'); + + // Check: Verify the theme is applied. + // We check that the button has the specific visual properties + // defined in our theme (gray color, xl radius). + await agent.check( + 'The "Test Button" has a gray background, indicating the theme\\'s primaryColor is active' + ); + await agent.check( + 'The "Test Button" has rounded corners, indicating the theme\\'s defaultRadius is active' + ); + + // Check: Verify the Paper component is rendered with its themed styles + await agent.check( + 'The page content is inside a "Paper" component with a border' + ); +}); diff --git a/docs/steps/step-03.md b/docs/steps/step-03.md new file mode 100644 index 0000000..b3e935a --- /dev/null +++ b/docs/steps/step-03.md @@ -0,0 +1,477 @@ +# **File: COMMIT\_03\_AUTH.md** + +## **Commit 3: ATproto OAuth Flow & SurrealDB JWT Generation** + +### **Objective** + +Implement the complete, high-risk ATproto OAuth flow. This is a complex, multi-step process: + +1. Resolve the user's handle to their PDS.9 +2. Discover the PDS's specific authorization\_endpoint.10 +3. Redirect the user to that endpoint. +4. Handle the callback, exchange the code for an ATproto token.11 +5. Use the ATproto token to get the user's canonical did. +6. Mint a new, *app-specific* SurrealDB JWT containing the did claim. +7. Set this SurrealDB JWT in a secure, httpOnly cookie for app session management. + +This commit directly addresses the user's identified **Risk 1**. + +### **Implementation Specification** + +**1\. Create lib/auth/atproto.ts** + +Create a file at /lib/auth/atproto.ts to abstract the complex ATproto discovery logic: + +TypeScript + +import { AtpAgent } from '@atproto/api'; + +/\*\* + \* Resolves a Bluesky handle (e.g., "user.bsky.social") to its + \* corresponding PDS (Personal Data Server) and DID (Decentralized Identifier). + \* This discovery step is mandatory before initiating OAuth. + \*/ +export async function resolveHandle(handle: string) { + try { + const agent \= new AtpAgent({ service: 'https://bsky.social' }); + const response \= await agent.resolveHandle({ handle }); + const did \= response.data.did; + + // Now, get the PDS from the DID document + const didDoc \= await agent.getDidDocument({ did }); + + // Find the 'atproto\_pds' service endpoint + const pdsService \= didDoc.service?.find( + (s) \=\> s.id \=== '\#atproto\_pds' + ); + + if (\!pdsService?.serviceEndpoint) { + throw new Error('PDS service endpoint not found in DID document.'); + } + + return { + did, + pdsUrl: pdsService.serviceEndpoint, + }; + } catch (error) { + console.error('Error resolving handle:', error); + throw new Error('Could not resolve Bluesky handle.'); + } +} + +/\*\* + \* Fetches the specific OAuth endpoints for a given PDS. + \* Each PDS has its own set of endpoints. + \*/ +export async function getAuthEndpoints(pdsUrl: string) { + try { + const metadataUrl \= \`${pdsUrl}/.well-known/oauth-authorization-server\`; + const response \= await fetch(metadataUrl); + + if (\!response.ok) { + throw new Error(\`Failed to fetch auth metadata from ${pdsUrl}\`); + } + + const metadata \= await response.json(); + + const { authorization\_endpoint, token\_endpoint } \= metadata; + + if (\!authorization\_endpoint ||\!token\_endpoint) { + throw new Error('Invalid auth metadata received from PDS.'); + } + + return { + authorizationEndpoint: authorization\_endpoint, + tokenEndpoint: token\_endpoint, + }; + } catch (error) { + console.error('Error getting auth endpoints:', error); + throw new Error('Could not discover OAuth endpoints.'); + } +} + +**2\. Create lib/auth/jwt.ts** + +Create a file at /lib/auth/jwt.ts to mint our app's internal JWT for SurrealDB: + +TypeScript + +import jwt from 'jsonwebtoken'; + +/\*\* + \* Mints a new JWT for our application's session management. + \* This token is what SurrealDB will validate. + \* + \* @param did \- The user's canonical ATproto DID (e.g., "did:plc:...") + \* @param handle \- The user's Bluesky handle (e.g., "user.bsky.social") + \* @returns A signed JWT string. + \*/ +export function mintSurrealJwt(did: string, handle: string): string { + const secret \= process.env.SURREALDB\_JWT\_SECRET; + if (\!secret) { + throw new Error('SURREALDB\_JWT\_SECRET is not set in environment.'); + } + + // This payload is critical. The \`did\` claim will be used + // in SurrealDB's PERMISSIONS clauses. + const payload \= { + // Standard JWT claims + iss: 'Ponderants', + aud: 'SurrealDB', + + // Custom claims + did: did, + handle: handle, + }; + + // Token expires in 7 days + const token \= jwt.sign(payload, secret, { + algorithm: 'HS512', + expiresIn: '7d', + }); + + return token; +} + +**3\. Create Login Page (app/login/page.tsx)** + +Create a file at /app/login/page.tsx: + +TypeScript + +'use client'; + +import { + Button, + Center, + Paper, + Stack, + TextInput, + Title, + Text, +} from '@mantine/core'; +import { useForm } from '@mantine/form'; +import { useState } from 'react'; +import { useRouter, useSearchParams } from 'next/navigation'; + +export default function LoginPage() { + const \[isLoading, setIsLoading\] \= useState(false); + const router \= useRouter(); + const searchParams \= useSearchParams(); + const error \= searchParams.get('error'); + + const form \= useForm({ + initialValues: { + handle: '', + }, + }); + + const handleSubmit \= async (values: { handle: string }) \=\> { + setIsLoading(true); + + // We redirect to our \*own\* API route, which will then + // perform discovery and redirect to the correct Bluesky PDS. + // This keeps all complex logic and secrets on the server. + router.push(\`/api/auth/login?handle=${values.handle}\`); + }; + + return ( + \<Center h="100vh"\> + \<Paper w={400} p="xl"\> + \<form onSubmit={form.onSubmit(handleSubmit)}\> + \<Stack\> + \<Title order={2} ta="center"\> + Log in to Ponderants + \</Title\> + \<Text ta="center" c="dimmed" size="sm"\> + Log in with your Bluesky handle. + \</Text\> + + {error && ( + \<Paper withBorder p="sm" bg="red.9"\> + \<Text c="white" size="sm"\> + Login Failed: {error} + \</Text\> + \</Paper\> + )} + + \<TextInput + label="Your Handle" + placeholder="e.g., yourname.bsky.social" + required + {...form.getInputProps('handle')} + /\> + \<Button type="submit" loading={isLoading} fullWidth\> + Log in with Bluesky + \</Button\> + \</Stack\> + \</form\> + \</Paper\> + \</Center\> + ); +} + +**4\. Create Auth Start Route (app/api/auth/login/route.ts)** + +Create a file at /app/api/auth/login/route.ts. This route starts the OAuth dance: + +TypeScript + +import { NextRequest, NextResponse } from 'next/server'; +import { cookies } from 'next/headers'; +import { getAuthEndpoints, resolveHandle } from '@/lib/auth/atproto'; +import { generators } from 'openid-client'; // For PKCE + +const CLIENT\_ID \= process.env.BLUESKY\_CLIENT\_ID; // e.g., 'https://ponderants.com/client-metadata.json' +const REDIRECT\_URI \= process.env.BLUESKY\_REDIRECT\_URI; // e.g., 'http://localhost:3000/api/auth/callback' + +export async function GET(request: NextRequest) { + if (\!CLIENT\_ID ||\!REDIRECT\_URI) { + throw new Error('Bluesky client configuration is missing.'); + } + + const { searchParams } \= new URL(request.url); + const handle \= searchParams.get('handle'); + + if (\!handle) { + return NextResponse.redirect(new URL('/login?error=Handle missing', request.url)); + } + + try { + // 1\. Resolve handle to get PDS + const { pdsUrl } \= await resolveHandle(handle); + + // 2\. Discover PDS-specific auth endpoints + const { authorizationEndpoint } \= await getAuthEndpoints(pdsUrl); + + // 3\. Generate PKCE challenge and state + const state \= generators.state(); + const code\_verifier \= generators.codeVerifier(); + const code\_challenge \= generators.codeChallenge(code\_verifier); + + // 4\. Store verifier and state in a temporary cookie + cookies().set('atproto\_oauth\_state', state, { httpOnly: true, maxAge: 600 }); + cookies().set('atproto\_pkce\_verifier', code\_verifier, { httpOnly: true, maxAge: 600 }); + cookies().set('atproto\_pds\_url', pdsUrl, { httpOnly: true, maxAge: 600 }); + + // 5\. Construct the authorization URL + const authUrl \= new URL(authorizationEndpoint); + authUrl.searchParams.set('response\_type', 'code'); + authUrl.searchParams.set('client\_id', CLIENT\_ID); + authUrl.searchParams.set('redirect\_uri', REDIRECT\_URI); + authUrl.searchParams.set('scope', 'atproto'); // Request full access + authUrl.searchParams.set('code\_challenge', code\_challenge); + authUrl.searchParams.set('code\_challenge\_method', 'S256'); + authUrl.searchParams.set('state', state); + + // 6\. Redirect user to the PDS login screen + return NextResponse.redirect(authUrl); + + } catch (error) { + console.error('Auth login error:', error); + return NextResponse.redirect(new URL('/login?error=Invalid handle or PDS', request.url)); + } +} + +**5\. Create Auth Callback Route (app/api/auth/callback/route.ts)** + +Create a file at /app/api/auth/callback/route.ts. This route handles the user's return: + +TypeScript + +import { NextRequest, NextResponse } from 'next/server'; +import { cookies } from 'next/headers'; +import { getAuthEndpoints } from '@/lib/auth/atproto'; +import { mintSurrealJwt } from '@/lib/auth/jwt'; +import { AtpAgent } from '@atproto/api'; + +const CLIENT\_ID \= process.env.BLUESKY\_CLIENT\_ID; +const REDIRECT\_URI \= process.env.BLUESKY\_REDIRECT\_URI; + +export async function GET(request: NextRequest) { + const { searchParams } \= new URL(request.url); + const code \= searchParams.get('code'); + const state \= searchParams.get('state'); + + // Get temporary values from cookies + const cookieState \= cookies().get('atproto\_oauth\_state')?.value; + const code\_verifier \= cookies().get('atproto\_pkce\_verifier')?.value; + const pdsUrl \= cookies().get('atproto\_pds\_url')?.value; + + // Clear temporary cookies + cookies().delete('atproto\_oauth\_state'); + cookies().delete('atproto\_pkce\_verifier'); + cookies().delete('atproto\_pds\_url'); + + // 1\. Validate state (CSRF protection) + if (\!state | + +| state\!== cookieState) { + return NextResponse.redirect(new URL('/login?error=Invalid state', request.url)); + } + + // 2\. Check for errors + if (\!code ||\!pdsUrl ||\!code\_verifier) { + return NextResponse.redirect(new URL('/login?error=Callback failed', request.url)); + } + + try { + // 3\. Get the PDS's token endpoint + const { tokenEndpoint } \= await getAuthEndpoints(pdsUrl); + + // 4\. Exchange the code for an ATproto access token + const tokenResponse \= await fetch(tokenEndpoint, { + method: 'POST', + headers: { 'Content-Type': 'application/x-www-form-urlencoded' }, + body: new URLSearchParams({ + grant\_type: 'authorization\_code', + code: code, + redirect\_uri: REDIRECT\_URI\!, + client\_id: CLIENT\_ID\!, + code\_verifier: code\_verifier, + }), + }); + + if (\!tokenResponse.ok) { + throw new Error('Failed to exchange code for token'); + } + + const { access\_token, refresh\_token } \= await tokenResponse.json(); + + // 5\. Use the ATproto token to get the user's session info (did, handle) + const agent \= new AtpAgent({ service: pdsUrl }); + agent.resumeSession({ + accessJwt: access\_token, + refreshJwt: refresh\_token, + did: '', // We don't know it yet + handle: '', // We don't know it yet + }); + + // getSession will populate the agent with the correct did/handle + const session \= await agent.getSession(); + + if (\!session.did ||\!session.handle) { + throw new Error('Failed to retrieve user session details'); + } + + // 6\. Mint OUR app's SurrealDB JWT + const surrealJwt \= mintSurrealJwt(session.did, session.handle); + + // 7\. Set the SurrealDB JWT in a secure cookie for our app + cookies().set('ponderants-auth', surrealJwt, { + httpOnly: true, + secure: process.env.NODE\_ENV \=== 'production', + maxAge: 60 \* 60 \* 24 \* 7, // 7 days + path: '/', + }); + + // (Hackathon Strategy: Store the ATproto tokens for later use. + // In production, this would go in an encrypted DB store.) + cookies().set('atproto\_access\_token', access\_token, { + httpOnly: true, secure: process.env.NODE\_ENV \=== 'production', maxAge: 60 \* 60, path: '/', + }); + cookies().set('atproto\_refresh\_token', refresh\_token, { + httpOnly: true, secure: process.env.NODE\_ENV \=== 'production', maxAge: 60 \* 60 \* 24 \* 30, path: '/', + }); + + // 8\. Redirect to the main application + return NextResponse.redirect(new URL('/chat', request.url)); + + } catch (error) { + console.error('Auth callback error:', error); + return NextResponse.redirect(new URL('/login?error=Internal auth error', request.url)); + } +} + +### **Test Specification** + +**1\. Create Test File (tests/magnitude/03-auth.mag.ts)** + +Create a file at /tests/magnitude/03-auth.mag.ts: + +TypeScript + +import { test } from 'magnitude-test'; + +test('\[Happy Path\] User can log in with Bluesky', async (agent) \=\> { + // Act: Go to login page + await agent.act('Navigate to /login'); + + // Act: Enter handle and submit + await agent.act( + 'Enter "testuser.bsky.social" into the "Your Handle" input' + ); + await agent.act('Click the "Log in with Bluesky" button'); + + // Check: Verify redirect to our API route + // (Magnitude will follow this redirect) + // We will mock the external OAuth flow. + // We simulate a successful return from Bluesky to our callback. + await agent.act( + 'Simulate a successful OAuth callback by navigating to the /api/auth/callback route with a valid code and state', + { + // This requires mocking the server's API routes to bypass the + // external fetch and PKCE/state verification. + // For the hackathon, we can test this by + // mocking the 'fetch' and 'cookies' calls. + + // A simpler E2E test for magnitude: + // 1\. Mock '/api/auth/login' to redirect to '/api/auth/callback' + // 2\. Mock '/api/auth/callback' to return a redirect to '/chat' + // and set the 'ponderants-auth' cookie. + + // Let's assume magnitude can check for redirects and cookies: + // The previous 'click' action on 'Log in' (which hits /api/auth/login) + // would be mocked to redirect to the external Bluesky URL. + // We can't follow that. + + // Let's test the callback handling directly. + // We assume the user has already been to Bluesky and is now + // hitting our callback. + url: '/api/auth/callback?code=mockcode\&state=mockstate', + + // We must mock the server-side logic in the test setup + // to validate 'mockstate' and 'mockcode' and return the + // final redirect \+ cookie. + } + ); + + // Check: User is redirected to the main app + await agent.check('The browser URL is now "http://localhost:3000/chat"'); + + // Check: The session cookie is set + await agent.check('A secure, httpOnly cookie named "ponderants-auth" is set'); +}); + +test('\[Unhappy Path\] User sees error on failed auth', async (agent) \=\> { + // Act: Simulate a failed callback from Bluesky + await agent.act( + 'Navigate to the /api/auth/callback route with an error', + { + url: '/api/auth/callback?error=invalid\_grant', + } + ); + + // Check: User is redirected back to the login page + await agent.check('The browser URL is now "http://localhost:3000/login"'); + + // Check: An error message is displayed + await agent.check('The text "Login Failed: Callback failed" is visible'); +}); + +test('\[Unhappy Path\] User sees error for invalid handle', async (agent) \=\> { + // Act: Go to login page + await agent.act('Navigate to /login'); + + // Act: Enter a non-existent handle + await agent.act( + 'Enter "nonexistent.handle.xyz" into the "Your Handle" input' + ); + + // (We mock the /api/auth/login route to fail resolution) + await agent.act('Click the "Log in with Bluesky" button'); + + // Check: User is redirected back to login with an error + await agent.check('The browser URL is now "http://localhost:3000/login"'); + await agent.check('The text "Login Failed: Invalid handle or PDS" is visible'); +}); diff --git a/docs/steps/step-04.md b/docs/steps/step-04.md new file mode 100644 index 0000000..09662f6 --- /dev/null +++ b/docs/steps/step-04.md @@ -0,0 +1,117 @@ +# **File: COMMIT\_04\_DB\_SCHEMA.md** + +## **Commit 4: SurrealDB Schema & Permissions** + +### **Objective** + +Define the SurrealDB "App View Cache" schemas. This includes: + +1. Defining trusted JWT access using our app's secret key. +2. Defining the user and node tables. +3. Defining the links\_to graph relation table. +4. **Crucially:** Implementing row-level security (PERMISSIONS) that uses the did claim from our JWT (minted in Commit 03\) to ensure users can *only* access their own data. +5. Defining the vector index for AI-powered linking. + +### **Implementation Specification** + +**1\. Create db/schema.surql** + +Create a file at /db/schema.surql. This script should be executed against the SurrealDB instance to initialize the schema. + +SQL + +\-- \-------------------------------------------------- +\-- Ponderants :: SurrealDB Schema +\-- \-------------------------------------------------- + +\-- \-------------------------------------------------- +\-- Access Control (JWT) +\-- \-------------------------------------------------- + +\-- Define the JWT access method. This tells SurrealDB to trust +\-- JWTs signed by our Next.js backend using the HS512 algorithm +\-- and the secret key provided in the environment. +\-- (Note: DEFINE TOKEN is deprecated as of 2.x) \[12\] +DEFINE ACCESS app\_jwt + ON DATABASE + TYPE JWT + ALGORITHM HS512 + KEY $env.SURREALDB\_JWT\_SECRET; \[13\] + +\-- \-------------------------------------------------- +\-- Table: user +\-- \-------------------------------------------------- + +\-- Stores basic user information, cached from ATproto. +DEFINE TABLE user SCHEMAFULL; + +\-- The user's decentralized identifier (DID) is their primary key. +DEFINE FIELD did ON TABLE user TYPE string + ASSERT $value\!= NONE; + +DEFINE FIELD handle ON TABLE user TYPE string; + +\-- Ensure DIDs are unique. +DEFINE INDEX user\_did\_idx ON TABLE user COLUMNS did UNIQUE; + +\-- \-------------------------------------------------- +\-- Table: node +\-- \-------------------------------------------------- + +\-- Stores a single "thought node." This is the cache record for +\-- the com.ponderants.node lexicon. +DEFINE TABLE node SCHEMAFULL + \-- THIS IS THE CORE SECURITY MODEL: + \-- Users can only perform actions on nodes where the + \-- node's 'user\_did' field matches the 'did' claim + \-- from their validated JWT ('$token.did'). \[12\] + PERMISSIONS + FOR select, create, update, delete + WHERE user\_did \= $token.did; + +\-- Foreign key linking to the user table (via DID). +DEFINE FIELD user\_did ON TABLE node TYPE string + ASSERT $value\!= NONE; + +\-- The canonical URI of the record on the ATproto PDS. +DEFINE FIELD atp\_uri ON TABLE node TYPE string; + +DEFINE FIELD title ON TABLE node TYPE string; +DEFINE FIELD body ON TABLE node TYPE string; + +\-- The AI-generated vector embedding for the 'body'. \[14\] +\-- We use array\<number\> for the vector. +DEFINE FIELD embedding ON TABLE node TYPE array\<number\>; + +\-- The 3D coordinates calculated by UMAP. +DEFINE FIELD coords\_3d ON TABLE node TYPE array\<number\> + \-- Must be a 3-point array \[x, y, z\] or empty. + ASSERT $value \= NONE OR array::len($value) \= 3; + +\-- Define the vector search index. +\-- We use MTREE (or HNSW) for high-performance k-NN search. +\-- The dimension (1536) MUST match the output of the +\-- 'gemini-embedding-001' model. +DEFINE INDEX node\_embedding\_idx ON TABLE node FIELDS embedding MTREE DIMENSION 1536; + +\-- \-------------------------------------------------- +\-- Relation: links\_to +\-- \-------------------------------------------------- + +\-- This is a graph edge table, relating (node)-\>(node). +DEFINE TABLE links\_to SCHEMAFULL + \-- Security for graph edges: A user can only create/view/delete + \-- links between two nodes that BOTH belong to them. + PERMISSIONS + FOR select, create, delete + WHERE + (SELECT user\_did FROM $from) \= $token.did + AND + (SELECT user\_did FROM $to) \= $token.did; + +\-- (No fields needed, it's a simple relation) +\-- Example usage: RELATE (node:1)-\[links\_to\]-\>(node:2); + +### **Test Specification** + +This commit contains no UI or API routes. The schemas defined here will be validated by the tests in subsequent commits (e.g., Commit 06 and Commit 10), which will attempt to write and read data according to these rules. diff --git a/docs/steps/step-05.md b/docs/steps/step-05.md new file mode 100644 index 0000000..1d89a18 --- /dev/null +++ b/docs/steps/step-05.md @@ -0,0 +1,67 @@ +# **File: COMMIT_05_LEXICON.md** + +## **Commit 5: ATproto Lexicon Definition** + +### **Objective** + +Define the custom ATproto Lexicon schema for com.ponderants.node. This JSON file +is the "Source of Truth" schema, defining the exact data shape that will be +published to the user's PDS. + +### **Implementation Specification** + +**1\. Create lexicon/com.ponderants.node.json** + +Create a file at /lexicon/com.ponderants.node.json. This file defines our +application's primary data type within the ATproto ecosystem.16 + +JSON + +{ + "lexicon": 1, + "id": "com.ponderants.node", + "defs": { + "main": { + "type": "record", + "description": "A Ponderants thought node. It represents a single, captured +idea, intended to be linked to other nodes to form a 'thought galaxy'.", + "record": { + "type": "object", + "required": \["createdAt", "title", "body"\], + "properties": { + "createdAt": { + "type": "string", + "format": "datetime" + }, + "title": { + "type": "string", + "maxLength": 256, + "description": "The title of the thought node." + }, + "body": { + "type": "string", + "maxLength": 3000, + "description": "The main content of the thought node, often generated by the AI +interviewer." + }, + "links": { + "type": "array", + "description": "An array of AT-URIs (as strong refs) pointing to other nodes +this node is linked to.", + "items": { + "type": "string", + "format": "at-uri" + }, + "maxLength": 50 + } + } + } + } + } +} + +### **Test Specification** + +This commit defines a static schema. It will be implicitly tested by the API in +Commit 06, which must successfully publish a record conforming to this exact +shape. diff --git a/docs/steps/step-06.md b/docs/steps/step-06.md new file mode 100644 index 0000000..8e7d279 --- /dev/null +++ b/docs/steps/step-06.md @@ -0,0 +1,202 @@ +# **File: COMMIT\_06\_WRITE\_FLOW.md** + +## **Commit 6: Core Write-Through Cache API** + +### **Objective** + +Implement the POST /api/nodes route. This is the core "write-through cache" logic, which is the architectural foundation of the application. It must: + +1. Authenticate the user via their SurrealDB JWT. +2. Retrieve their ATproto access token (from the encrypted cookie). +3. **Step 1 (Truth):** Publish the new node to their PDS using the com.ponderants.node lexicon. +4. **Step 2 (Cache):** Generate a gemini-embedding-001 vector from the node's body. +5. **Step 3 (Cache):** Write the node, its atp\_uri, and its embedding to our SurrealDB cache. + +### **Implementation Specification** + +**1\. Create lib/db.ts** + +Create a helper file at /lib/db.ts for connecting to SurrealDB: + +TypeScript + +import { Surreal } from 'surrealdb.js'; + +const db \= new Surreal(); + +/\*\* + \* Connects to the SurrealDB instance. + \* @param {string} token \- The user's app-specific (SurrealDB) JWT. + \*/ +export async function connectToDB(token: string) { + if (\!db.connected) { + await db.connect(process.env.SURREALDB\_URL\!); + } + + // Authenticate as the user for this request. + // This enforces the row-level security (PERMISSIONS) + // defined in the schema for all subsequent queries. + await db.authenticate(token); + + return db; +} + +**2\. Create lib/ai.ts** + +Create a helper file at /lib/ai.ts for AI operations: + +TypeScript + +import { GoogleGenerativeAI } from '@google/generative-ai'; + +const genAI \= new GoogleGenerativeAI(process.env.GOOGLE\_API\_KEY\!); + +const embeddingModel \= genAI.getGenerativeModel({ + model: 'gemini-embedding-001', +}); + +/\*\* + \* Generates a vector embedding for a given text. + \* @param text The text to embed. + \* @returns A 1536-dimension vector (Array\<number\>). + \*/ +export async function generateEmbedding(text: string): Promise\<number\> { + try { + const result \= await embeddingModel.embedContent(text); + return result.embedding.values; + } catch (error) { + console.error('Error generating embedding:', error); + throw new Error('Failed to generate AI embedding.'); + } +} + +**3\. Create Write API Route (app/api/nodes/route.ts)** + +Create the main API file at /app/api/nodes/route.ts: + +TypeScript + +import { NextRequest, NextResponse } from 'next/server'; +import { cookies } from 'next/headers'; +import { AtpAgent, RichText } from '@atproto/api'; +import { connectToDB } from '@/lib/db'; +import { generateEmbedding } from '@/lib/ai'; + +export async function POST(request: NextRequest) { + const surrealJwt \= cookies().get('ponderants-auth')?.value; + const atpAccessToken \= cookies().get('atproto\_access\_token')?.value; + + if (\!surrealJwt ||\!atpAccessToken) { + return NextResponse.json({ error: 'Not authenticated' }, { status: 401 }); + } + + let userDid: string; + try { + // Decode the JWT to get the DID for the SurrealDB query + // In a real app, we'd verify it, but for now we just + // pass it to connectToDB which authenticates with it. + const { payload } \= jwt.decode(surrealJwt, { complete: true })\!; + userDid \= (payload as { did: string }).did; + } catch (e) { + return NextResponse.json({ error: 'Invalid auth token' }, { status: 401 }); + } + + const { title, body, links } \= (await request.json()) as { + title: string; + body: string; + links: string; // Array of at-uri strings + }; + + if (\!title ||\!body) { + return NextResponse.json({ error: 'Title and body are required' }, { status: 400 }); + } + + const createdAt \= new Date().toISOString(); + + // \--- Step 1: Write to Source of Truth (ATproto) \--- + let atp\_uri: string; + let atp\_cid: string; + + try { + const agent \= new AtpAgent({ service: 'https://bsky.social' }); // The service URL may need to be dynamic + await agent.resumeSession({ accessJwt: atpAccessToken, did: userDid, handle: '' }); // Simplified resume + + // Format the body as RichText + const rt \= new RichText({ text: body }); + await rt.detectFacets(agent); // Detect links, mentions + + const response \= await agent.post({ + $type: 'com.ponderants.node', + repo: userDid, + collection: 'com.ponderants.node', + record: { + title, + body: rt.text, + facets: rt.facets, // Include facets for rich text + links: links?.map(uri \=\> ({ $link: uri })) ||, // Convert URIs to strong refs + createdAt, + }, + }); + + atp\_uri \= response.uri; + atp\_cid \= response.cid; + + } catch (error) { + console.error('ATproto write error:', error); + return NextResponse.json({ error: 'Failed to publish to PDS' }, { status: 500 }); + } + + // \--- Step 2: Generate AI Embedding (Cache) \--- + let embedding: number; + try { + embedding \= await generateEmbedding(title \+ '\\n' \+ body); + } catch (error) { + console.error('Embedding error:', error); + return NextResponse.json({ error: 'Failed to generate embedding' }, { status: 500 }); + } + + // \--- Step 3: Write to App View Cache (SurrealDB) \--- + try { + const db \= await connectToDB(surrealJwt); + + // Create the node record in our cache. + // The \`user\_did\` field is set, satisfying the 'PERMISSIONS' + // clause defined in the schema. + const newNode \= await db.create('node', { + user\_did: userDid, + atp\_uri: atp\_uri, + title: title, + body: body, // Store the raw text body + embedding: embedding, + // coords\_3d will be calculated later + }); + + // Handle linking + if (links && links.length \> 0) { + // Find the corresponding cache nodes for the AT-URIs + const targetNodes: { id: string } \= await db.query( + 'SELECT id FROM node WHERE user\_did \= $did AND atp\_uri IN $links', + { did: userDid, links: links } + ); + + // Create graph relations + for (const targetNode of targetNodes) { + await db.query('RELATE $from-\>links\_to-\>$to', { + from: (newNode as any).id, + to: targetNode.id, + }); + } + } + + return NextResponse.json(newNode); + + } catch (error) { + console.error('SurrealDB write error:', error); + // TODO: Implement rollback for the ATproto post? + return NextResponse.json({ error: 'Failed to save to app cache' }, { status: 500 }); + } +} + +### **Test Specification** + +This is an API-only commit. It will be tested via the end-to-end flow in **Commit 10 (Linking)**, which will provide the UI (the "Publish" button) to trigger this route. diff --git a/docs/steps/step-07.md b/docs/steps/step-07.md new file mode 100644 index 0000000..8e7d279 --- /dev/null +++ b/docs/steps/step-07.md @@ -0,0 +1,202 @@ +# **File: COMMIT\_06\_WRITE\_FLOW.md** + +## **Commit 6: Core Write-Through Cache API** + +### **Objective** + +Implement the POST /api/nodes route. This is the core "write-through cache" logic, which is the architectural foundation of the application. It must: + +1. Authenticate the user via their SurrealDB JWT. +2. Retrieve their ATproto access token (from the encrypted cookie). +3. **Step 1 (Truth):** Publish the new node to their PDS using the com.ponderants.node lexicon. +4. **Step 2 (Cache):** Generate a gemini-embedding-001 vector from the node's body. +5. **Step 3 (Cache):** Write the node, its atp\_uri, and its embedding to our SurrealDB cache. + +### **Implementation Specification** + +**1\. Create lib/db.ts** + +Create a helper file at /lib/db.ts for connecting to SurrealDB: + +TypeScript + +import { Surreal } from 'surrealdb.js'; + +const db \= new Surreal(); + +/\*\* + \* Connects to the SurrealDB instance. + \* @param {string} token \- The user's app-specific (SurrealDB) JWT. + \*/ +export async function connectToDB(token: string) { + if (\!db.connected) { + await db.connect(process.env.SURREALDB\_URL\!); + } + + // Authenticate as the user for this request. + // This enforces the row-level security (PERMISSIONS) + // defined in the schema for all subsequent queries. + await db.authenticate(token); + + return db; +} + +**2\. Create lib/ai.ts** + +Create a helper file at /lib/ai.ts for AI operations: + +TypeScript + +import { GoogleGenerativeAI } from '@google/generative-ai'; + +const genAI \= new GoogleGenerativeAI(process.env.GOOGLE\_API\_KEY\!); + +const embeddingModel \= genAI.getGenerativeModel({ + model: 'gemini-embedding-001', +}); + +/\*\* + \* Generates a vector embedding for a given text. + \* @param text The text to embed. + \* @returns A 1536-dimension vector (Array\<number\>). + \*/ +export async function generateEmbedding(text: string): Promise\<number\> { + try { + const result \= await embeddingModel.embedContent(text); + return result.embedding.values; + } catch (error) { + console.error('Error generating embedding:', error); + throw new Error('Failed to generate AI embedding.'); + } +} + +**3\. Create Write API Route (app/api/nodes/route.ts)** + +Create the main API file at /app/api/nodes/route.ts: + +TypeScript + +import { NextRequest, NextResponse } from 'next/server'; +import { cookies } from 'next/headers'; +import { AtpAgent, RichText } from '@atproto/api'; +import { connectToDB } from '@/lib/db'; +import { generateEmbedding } from '@/lib/ai'; + +export async function POST(request: NextRequest) { + const surrealJwt \= cookies().get('ponderants-auth')?.value; + const atpAccessToken \= cookies().get('atproto\_access\_token')?.value; + + if (\!surrealJwt ||\!atpAccessToken) { + return NextResponse.json({ error: 'Not authenticated' }, { status: 401 }); + } + + let userDid: string; + try { + // Decode the JWT to get the DID for the SurrealDB query + // In a real app, we'd verify it, but for now we just + // pass it to connectToDB which authenticates with it. + const { payload } \= jwt.decode(surrealJwt, { complete: true })\!; + userDid \= (payload as { did: string }).did; + } catch (e) { + return NextResponse.json({ error: 'Invalid auth token' }, { status: 401 }); + } + + const { title, body, links } \= (await request.json()) as { + title: string; + body: string; + links: string; // Array of at-uri strings + }; + + if (\!title ||\!body) { + return NextResponse.json({ error: 'Title and body are required' }, { status: 400 }); + } + + const createdAt \= new Date().toISOString(); + + // \--- Step 1: Write to Source of Truth (ATproto) \--- + let atp\_uri: string; + let atp\_cid: string; + + try { + const agent \= new AtpAgent({ service: 'https://bsky.social' }); // The service URL may need to be dynamic + await agent.resumeSession({ accessJwt: atpAccessToken, did: userDid, handle: '' }); // Simplified resume + + // Format the body as RichText + const rt \= new RichText({ text: body }); + await rt.detectFacets(agent); // Detect links, mentions + + const response \= await agent.post({ + $type: 'com.ponderants.node', + repo: userDid, + collection: 'com.ponderants.node', + record: { + title, + body: rt.text, + facets: rt.facets, // Include facets for rich text + links: links?.map(uri \=\> ({ $link: uri })) ||, // Convert URIs to strong refs + createdAt, + }, + }); + + atp\_uri \= response.uri; + atp\_cid \= response.cid; + + } catch (error) { + console.error('ATproto write error:', error); + return NextResponse.json({ error: 'Failed to publish to PDS' }, { status: 500 }); + } + + // \--- Step 2: Generate AI Embedding (Cache) \--- + let embedding: number; + try { + embedding \= await generateEmbedding(title \+ '\\n' \+ body); + } catch (error) { + console.error('Embedding error:', error); + return NextResponse.json({ error: 'Failed to generate embedding' }, { status: 500 }); + } + + // \--- Step 3: Write to App View Cache (SurrealDB) \--- + try { + const db \= await connectToDB(surrealJwt); + + // Create the node record in our cache. + // The \`user\_did\` field is set, satisfying the 'PERMISSIONS' + // clause defined in the schema. + const newNode \= await db.create('node', { + user\_did: userDid, + atp\_uri: atp\_uri, + title: title, + body: body, // Store the raw text body + embedding: embedding, + // coords\_3d will be calculated later + }); + + // Handle linking + if (links && links.length \> 0) { + // Find the corresponding cache nodes for the AT-URIs + const targetNodes: { id: string } \= await db.query( + 'SELECT id FROM node WHERE user\_did \= $did AND atp\_uri IN $links', + { did: userDid, links: links } + ); + + // Create graph relations + for (const targetNode of targetNodes) { + await db.query('RELATE $from-\>links\_to-\>$to', { + from: (newNode as any).id, + to: targetNode.id, + }); + } + } + + return NextResponse.json(newNode); + + } catch (error) { + console.error('SurrealDB write error:', error); + // TODO: Implement rollback for the ATproto post? + return NextResponse.json({ error: 'Failed to save to app cache' }, { status: 500 }); + } +} + +### **Test Specification** + +This is an API-only commit. It will be tested via the end-to-end flow in **Commit 10 (Linking)**, which will provide the UI (the "Publish" button) to trigger this route. diff --git a/docs/steps/step-08.md b/docs/steps/step-08.md new file mode 100644 index 0000000..2fa6866 --- /dev/null +++ b/docs/steps/step-08.md @@ -0,0 +1,73 @@ +# **File: COMMIT\_08\_VOICE\_TOKEN.md** + +## **Commit 8: Real-time Voice: Token API** + +### **Objective** + +Implement the serverless-friendly Deepgram token generation API. This is the expert-level correction to the user's "WebSocket proxy" plan. A standard Vercel deployment does not support persistent WebSocket backends.22 + +This architecture (based on the nextjs-live-transcription starter's implications 23) provides a secure, stateless way to use Deepgram's real-time streaming: + +1. The client requests a temporary token from our Next.js API. +2. Our Next.js API (server-side) uses + our secret Deepgram API key to generate a temporary, short-lived token. +3. The client receives this temporary token and uses it to connect *directly* to Deepgram's WebSocket endpoint. + +### **Implementation Specification** + +**1\. Create app/api/voice-token/route.ts** + +Create a file at /app/api/voice-token/route.ts: + +TypeScript + +import { NextRequest, NextResponse } from 'next/server'; +import { DeepgramClient, createClient } from '@deepgram/sdk'; + +/\*\* + \* This API route generates a short-lived, temporary API key + \* for a client to connect directly to Deepgram's WebSocket. + \* This avoids exposing our main API key and bypasses + \* serverless WebSocket limitations. + \*/ +export async function POST(request: NextRequest) { + const deepgramApiKey \= process.env.DEEPGRAM\_API\_KEY; + + if (\!deepgramApiKey) { + return NextResponse.json( + { error: 'Deepgram API key not configured' }, + { status: 500 } + ); + } + + const deepgram: DeepgramClient \= createClient(deepgramApiKey); + + try { + // Create a new, temporary key with 'member' permissions + // that expires in 1 minute (60 seconds). + const { key, error } \= await deepgram.keys.create( + null, // Let Deepgram generate the key + 'Temporary key for Ponderants user', + \['member'\], // Required scope for transcription + { timeToLive: 60 } + ); + + if (error) { + throw error; + } + + // Send the temporary key back to the client + return NextResponse.json({ key }); + + } catch (error) { + console.error('Error creating Deepgram key:', error); + return NextResponse.json( + { error: 'Failed to generate voice token' }, + { status: 500 } + ); + } +} + +### **Test Specification** + +This is an API-only commit. It will be tested by the client implemented in **Commit 09**. diff --git a/docs/steps/step-09.md b/docs/steps/step-09.md new file mode 100644 index 0000000..d8cc841 --- /dev/null +++ b/docs/steps/step-09.md @@ -0,0 +1,296 @@ +# **File: COMMIT\_09\_VOICE\_CLIENT.md** + +## **Commit 9: Real-time Voice: Client Integration** + +### **Objective** + +Integrate the client-side microphone recording. This component will: + +1. Call the /api/voice-token route (from Commit 08\) to get a temporary key. +2. Use navigator.mediaDevices.getUserMedia to access the microphone.24 +3. Open a *direct* WebSocket to Deepgram using the temporary key.25 +4. Use MediaRecorder to capture audio chunks.24 +5. Stream audio chunks to Deepgram and receive transcripts back in real-time. +6. Feed the received transcripts into the useChat input. + +### **Implementation Specification** + +**1\. Create components/MicrophoneRecorder.tsx** + +Create a new component at /components/MicrophoneRecorder.tsx: + +TypeScript + +'use client'; + +import { ActionIcon, Tooltip } from '@mantine/core'; +import { IconMicrophone, IconMicrophoneOff } from '@tabler/icons-react'; +import { useState, useRef } from 'react'; + +// Define the shape of the Deepgram transcript +interface DeepgramTranscript { + channel: { + alternatives: { + transcript: string; + }; + }; + is\_final: boolean; + speech\_final: boolean; +} + +type Props \= { + /\*\* + \* Callback function to update the chat input with the new transcript. + \* @param transcript The full, combined transcript. + \*/ + onTranscriptUpdate: (transcript: string) \=\> void; + /\*\* + \* Callback function to signal the final transcript for this "thought". + \* @param transcript The final, punctuated transcript. + \*/ + onTranscriptFinalized: (transcript: string) \=\> void; +}; + +export function MicrophoneRecorder({ onTranscriptUpdate, onTranscriptFinalized }: Props) { + const \= useState(false); + const mediaRecorderRef \= useRef\<MediaRecorder | null\>(null); + const socketRef \= useRef\<WebSocket | null\>(null); + + // Store the combined transcript for the current utterance + const transcriptRef \= useRef\<string\>(''); + + const stopRecording \= () \=\> { + if (mediaRecorderRef.current) { + mediaRecorderRef.current.stop(); + mediaRecorderRef.current \= null; + } + if (socketRef.current) { + socketRef.current.close(); + socketRef.current \= null; + } + setIsRecording(false); + + // Finalize the transcript + if (transcriptRef.current) { + onTranscriptFinalized(transcriptRef.current); + } + transcriptRef.current \= ''; + }; + + const startRecording \= async () \=\> { + transcriptRef.current \= ''; // Reset transcript + try { + // 1\. Get the temporary Deepgram key + const response \= await fetch('/api/voice-token', { method: 'POST' }); + const { key, error } \= await response.json(); + + if (error) { + throw new Error(error); + } + + // 2\. Access the microphone + const stream \= await navigator.mediaDevices.getUserMedia({ audio: true }); + + // 3\. Open direct WebSocket to Deepgram + const socket \= new WebSocket( + 'wss://api.deepgram.com/v1/listen?interim\_results=true\&punctuate=true', + \['token', key\] + ); + socketRef.current \= socket; + + socket.onopen \= () \=\> { + // 4\. Create MediaRecorder + const mediaRecorder \= new MediaRecorder(stream, { + mimeType: 'audio/webm', + }); + mediaRecorderRef.current \= mediaRecorder; + + // 5\. Send audio chunks on data available + mediaRecorder.ondataavailable \= (event) \=\> { + if (event.data.size \> 0 && socket.readyState \=== WebSocket.OPEN) { + socket.send(event.data); + } + }; + + // Start recording and chunking audio every 250ms + mediaRecorder.start(250); + setIsRecording(true); + }; + + // 6\. Receive transcripts + socket.onmessage \= (event) \=\> { + const data \= JSON.parse(event.data) as DeepgramTranscript; + const transcript \= data.channel.alternatives.transcript; + + if (transcript) { + transcriptRef.current \= transcript; + onTranscriptUpdate(transcript); + } + + // If it's a "speech final" event, this utterance is done. + if (data.speech\_final) { + stopRecording(); + } + }; + + socket.onclose \= () \=\> { + // Clean up stream + stream.getTracks().forEach((track) \=\> track.stop()); + if (isRecording) { + stopRecording(); // Ensure cleanup + } + }; + + socket.onerror \= (err) \=\> { + console.error('WebSocket error:', err); + stopRecording(); + }; + + } catch (error) { + console.error('Error starting recording:', error); + setIsRecording(false); + } + }; + + const handleToggleRecord \= () \=\> { + if (isRecording) { + stopRecording(); + } else { + startRecording(); + } + }; + + return ( + \<Tooltip label={isRecording? 'Stop Recording' : 'Start Recording'}\> + \<ActionIcon + onClick={handleToggleRecord} + size="lg" + radius="xl" + color={isRecording? 'red' : 'gray'} + variant="filled" + \> + {isRecording? \<IconMicrophoneOff /\> : \<IconMicrophone /\>} + \</ActionIcon\> + \</Tooltip\> + ); +} + +**2\. Update Chat UI (app/chat/page.tsx)** + +Update /app/chat/page.tsx to include the new component: + +TypeScript + +'use client'; + +//... (other imports) +import { MicrophoneRecorder } from '@/components/MicrophoneRecorder'; + +export default function ChatPage() { + //... (other hooks: router, viewport) + + const { + messages, + input, + handleInputChange, + handleSubmit, + setInput, // Get the setInput setter from useChat + data, + isLoading, + } \= useChat({ + //... (rest of useChat config) + }); + + //... (useEffect for auto-scroll) + + return ( + \<Container size="md" h="100vh" style={{ display: 'flex', flexDirection: 'column' }}\> + {/\*... (Title and ScrollArea)... \*/} + + \<form onSubmit={handleSubmit}\> + \<Paper withBorder p="sm" radius="xl" my="md"\> + \<Group\> + \<TextInput + value={input} + onChange={handleInputChange} + placeholder="Speak or type your thoughts..." + style={{ flex: 1 }} + variant="unstyled" + disabled={isLoading} + /\> + + {/\* Add the Microphone Recorder Here \*/} + \<MicrophoneRecorder + onTranscriptUpdate={(transcript) \=\> { + // Update the input field in real-time + setInput(transcript); + }} + onTranscriptFinalized={(transcript) \=\> { + // Automatically submit the chat when speech is final + // We pass the final transcript in the options + handleSubmit(new Event('submit'), { + data: { + finalTranscript: transcript, + }, + }); + }} + /\> + + \<Button type="submit" radius="xl" loading={isLoading}\> + Send + \</Button\> + \</Group\> + \</Paper\> + \</form\> + \</Container\> + ); +} + +### **Test Specification** + +**1\. Create Test File (tests/magnitude/09-voice.mag.ts)** + +Create a file at /tests/magnitude/09-voice.mag.ts: + +TypeScript + +import { test } from 'magnitude-test'; + +test('\[Happy Path\] User can record voice and see transcript', async (agent) \=\> { + // Act: Go to chat page + await agent.act('Navigate to /chat'); + + // Check: Verify initial state + await agent.check('The chat input field is empty'); + await agent.check('A "Start Recording" button is visible'); + + // Act: Click the record button + // We must mock the /api/voice-token response and the + // MediaDevices/WebSocket browser APIs. + await agent.act('Click the "Start Recording" button'); + + // Check: UI updates to recording state + await agent.check('A "Stop Recording" button is visible'); + + // Act: Simulate receiving a transcript from the (mocked) Deepgram WebSocket + await agent.act( + 'Simulate an interim transcript "Hello world" from the Deepgram WebSocket' + ); + + // Check: The input field is updated + await agent.check('The chat input field contains "Hello world"'); + + // Act: Simulate a final transcript + await agent.act( + 'Simulate a final transcript "Hello world." from the Deepgram WebSocket' + ); + + // Check: The "Stop Recording" button is gone + await agent.check('A "Start Recording" button is visible again'); + + // Check: The chat input is cleared (because it was submitted) + await agent.check('The chat input field is empty'); + + // Check: The finalized transcript appears as a user message + await agent.check('The message "Hello world." appears in the chat list'); +}); diff --git a/docs/steps/step-10.md b/docs/steps/step-10.md new file mode 100644 index 0000000..ca69062 --- /dev/null +++ b/docs/steps/step-10.md @@ -0,0 +1,346 @@ +# **File: COMMIT\_10\_LINKING.md** + +## **Commit 10: Node Editor & AI-Powered Linking** + +### **Objective** + +Build the node editor UI and the AI-powered "Find related" feature. This commit will: + +1. Create the editor page (/editor/\[id\]) that is pre-filled by the chat (Commit 07\) or loaded from the DB. +2. Implement the "Publish" button, which calls the /api/nodes route (from Commit 06). +3. Implement the "Find related" button, which calls a *new* /api/suggest-links route. +4. Implement the /api/suggest-links route, which generates an embedding for the current draft and uses SurrealDB's vector search to find similar nodes.15 + +### **Implementation Specification** + +**1\. Create Editor Page (app/editor/\[id\]/page.tsx)** + +Create a file at /app/editor/\[id\]/page.tsx: + +TypeScript + +'use client'; + +import { + Container, + Title, + TextInput, + Textarea, + Button, + Stack, + Paper, + Text, + LoadingOverlay, + Group, +} from '@mantine/core'; +import { useForm } from '@mantine/form'; +import { useSearchParams, useRouter, useParams } from 'next/navigation'; +import { useEffect, useState } from 'react'; + +// Define the shape of a suggested link +interface SuggestedNode { + id: string; + title: string; + body: string; + score: number; +} + +export default function EditorPage() { + const router \= useRouter(); + const params \= useParams(); + const searchParams \= useSearchParams(); + + const \[isPublishing, setIsPublishing\] \= useState(false); + const \[isFinding, setIsFinding\] \= useState(false); + const \= useState\<SuggestedNode\>(); + + const form \= useForm({ + initialValues: { + title: '', + body: '', + links: as string, // Array of at-uri strings + }, + }); + + // Pre-fill form from search params (from AI chat redirect) + useEffect(() \=\> { + if (params.id \=== 'new') { + const title \= searchParams.get('title') | + +| ''; + const body \= searchParams.get('body') | + +| ''; + form.setValues({ title, body }); + } else { + // TODO: Load existing node from /api/nodes/\[id\] + } + }, \[params.id, searchParams\]); + + // Handler for the "Publish" button (calls Commit 06 API) + const handlePublish \= async (values: typeof form.values) \=\> { + setIsPublishing(true); + try { + const response \= await fetch('/api/nodes', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify(values), + }); + + if (\!response.ok) { + throw new Error('Failed to publish node'); + } + + const newNode \= await response.json(); + // On success, go to the graph + router.push('/galaxy'); + + } catch (error) { + console.error(error); + // TODO: Show notification + } finally { + setIsPublishing(false); + } + }; + + // Handler for the "Find related" button + const handleFindRelated \= async () \=\> { + setIsFinding(true); + setSuggestions(); + try { + const response \= await fetch('/api/suggest-links', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ body: form.values.body }), + }); + + if (\!response.ok) { + throw new Error('Failed to find links'); + } + + const relatedNodes \= await response.json(); + setSuggestions(relatedNodes); + + } catch (error) { + console.error(error); + // TODO: Show notification + } finally { + setIsFinding(false); + } + }; + + return ( + \<Container size="md" py="xl"\> + \<form onSubmit={form.onSubmit(handlePublish)}\> + \<Stack gap="md"\> + \<Title order={2}\> + {params.id \=== 'new'? 'Create New Node' : 'Edit Node'} + \</Title\> + + \<TextInput + label="Title" + placeholder="Your node title" + required + {...form.getInputProps('title')} + /\> + + \<Textarea + label="Body" + placeholder="Your node content..." + required + minRows={10} + autosize + {...form.getInputProps('body')} + /\> + + \<Group\> + \<Button + type="button" + variant="outline" + onClick={handleFindRelated} + loading={isFinding} + \> + Find Related + \</Button\> + \<Button type="submit" loading={isPublishing}\> + Publish Node + \</Button\> + \</Group\> + + {/\* Related Links Section \*/} + \<Stack\> + {isFinding && \<LoadingOverlay visible /\>} + {suggestions.length \> 0 && \<Title order={4}\>Suggested Links\</Title\>} + + {suggestions.map((node) \=\> ( + \<Paper key={node.id} withBorder p="sm"\> + \<Text fw={700}\>{node.title}\</Text\> + \<Text size="sm" lineClamp={2}\>{node.body}\</Text\> + \<Text size="xs" c="dimmed"\>Similarity: {(node.score \* 100).toFixed(0)}%\</Text\> + \</Paper\> + ))} + + {\!isFinding && suggestions.length \=== 0 && ( + \<Text size="sm" c="dimmed"\> + {/\* Placeholder text \*/} + \</Text\> + )} + \</Stack\> + + \</Stack\> + \</form\> + \</Container\> + ); +} + +**2\. Create Link Suggestion API (app/api/suggest-links/route.ts)** + +Create a file at /app/api/suggest-links/route.ts: + +TypeScript + +import { NextRequest, NextResponse } from 'next/server'; +import { cookies } from 'next/headers'; +import { connectToDB } from '@/lib/db'; +import { generateEmbedding } from '@/lib/ai'; + +export async function POST(request: NextRequest) { + const surrealJwt \= cookies().get('ponderants-auth')?.value; + if (\!surrealJwt) { + return NextResponse.json({ error: 'Not authenticated' }, { status: 401 }); + } + + const { body } \= await request.json(); + if (\!body) { + return NextResponse.json({ error: 'Body text is required' }, { status: 400 }); + } + + try { + // 1\. Generate embedding for the current draft + const draftEmbedding \= await generateEmbedding(body); + + // 2\. Connect to DB (as the user) + const db \= await connectToDB(surrealJwt); + + // 3\. Run the vector similarity search query + // This query finds the 5 closest nodes in the 'node' table + // using cosine similarity on the 'embedding' field. + // It only searches nodes WHERE user\_did \= $token.did, + // which is enforced by the table's PERMISSIONS. + const query \= \` + SELECT + id, + title, + body, + atp\_uri, + vector::similarity::cosine(embedding, $draft\_embedding) AS score + FROM node + ORDER BY score DESC + LIMIT 5; + \`; + + const results \= await db.query(query, { + draft\_embedding: draftEmbedding, + }); + + // The query returns an array of result sets. We want the first one. + return NextResponse.json(results.result); + + } catch (error) { + console.error('Link suggestion error:', error); + return NextResponse.json( + { error: 'Failed to suggest links' }, + { status: 500 } + ); + } +} + +### **Test Specification** + +**1\. Create Test File (tests/magnitude/10-linking.mag.ts)** + +Create a file at /tests/magnitude/10-linking.mag.ts: + +TypeScript + +import { test } from 'magnitude-test'; + +// Helper function to seed the database for this test +async function seedDatabase(agent) { + // This would use a custom magnitude.run command or API + // to pre-populate the SurrealDB instance with mock nodes. + await agent.act('Seed the database with 3 nodes: "Node A", "Node B", "Node C"'); + // "Node A" is about "dogs and cats" + // "Node B" is about "vector databases" + // "Node C" is about "ATproto" +} + +test('\[Happy Path\] User can find related links for a draft', async (agent) \=\> { + // Setup: Seed the DB + await seedDatabase(agent); + + // Act: Navigate to the editor + await agent.act('Navigate to /editor/new'); + + // Act: Fill out the form with a related idea + await agent.act( + 'Enter "My New Post" into the "Title" input' + ); + await agent.act( + 'Enter "This idea is about vectors and databases, and how they work." into the "Body" textarea' + ); + + // Act: Click the find related button + // (Mock the /api/suggest-links route to return "Node B") + await agent.act('Click the "Find Related" button'); + + // Check: The related node appears in the suggestions + await agent.check('A list of suggested links appears'); + await agent.check('The suggested node "Node B" is visible in the list'); + await agent.check('The suggested node "Node A" is NOT visible in the list'); +}); + +test('\[Unhappy Path\] User sees empty state when no links found', async (agent) \=\> { + // Setup: Seed the DB + await seedDatabase(agent); + + // Act: Navigate to the editor + await agent.act('Navigate to /editor/new'); + + // Act: Fill out the form with an unrelated idea + await agent.act( + 'Enter "Zebras" into the "Title" input' + ); + await agent.act( + 'Enter "Zebras are striped equines." into the "Body" textarea' + ); + + // Act: Click the find related button + // (Mock the /api/suggest-links route to return an empty array) + await agent.act('Click the "Find Related" button'); + + // Check: An empty state is shown + await agent.check('The text "No related nodes found" is visible'); +}); + +test('\[Happy Path\] User can publish a new node', async (agent) \=\> { + // Act: Navigate to the editor + await agent.act('Navigate to /editor/new'); + + // Act: Fill out the form + await agent.act( + 'Enter "My First Published Node" into the "Title" input' + ); + await agent.act( + 'Enter "This is the body of my first node." into the "Body" textarea' + ); + + // Act: Click Publish + // (Mock the /api/nodes route (Commit 06\) to return success) + await agent.act('Click the "Publish Node" button'); + + // Check: User is redirected to the galaxy + await agent.check( + 'The browser URL is now "http://localhost:3000/galaxy"' + ); +}); diff --git a/docs/steps/step-11.md b/docs/steps/step-11.md new file mode 100644 index 0000000..ca69062 --- /dev/null +++ b/docs/steps/step-11.md @@ -0,0 +1,346 @@ +# **File: COMMIT\_10\_LINKING.md** + +## **Commit 10: Node Editor & AI-Powered Linking** + +### **Objective** + +Build the node editor UI and the AI-powered "Find related" feature. This commit will: + +1. Create the editor page (/editor/\[id\]) that is pre-filled by the chat (Commit 07\) or loaded from the DB. +2. Implement the "Publish" button, which calls the /api/nodes route (from Commit 06). +3. Implement the "Find related" button, which calls a *new* /api/suggest-links route. +4. Implement the /api/suggest-links route, which generates an embedding for the current draft and uses SurrealDB's vector search to find similar nodes.15 + +### **Implementation Specification** + +**1\. Create Editor Page (app/editor/\[id\]/page.tsx)** + +Create a file at /app/editor/\[id\]/page.tsx: + +TypeScript + +'use client'; + +import { + Container, + Title, + TextInput, + Textarea, + Button, + Stack, + Paper, + Text, + LoadingOverlay, + Group, +} from '@mantine/core'; +import { useForm } from '@mantine/form'; +import { useSearchParams, useRouter, useParams } from 'next/navigation'; +import { useEffect, useState } from 'react'; + +// Define the shape of a suggested link +interface SuggestedNode { + id: string; + title: string; + body: string; + score: number; +} + +export default function EditorPage() { + const router \= useRouter(); + const params \= useParams(); + const searchParams \= useSearchParams(); + + const \[isPublishing, setIsPublishing\] \= useState(false); + const \[isFinding, setIsFinding\] \= useState(false); + const \= useState\<SuggestedNode\>(); + + const form \= useForm({ + initialValues: { + title: '', + body: '', + links: as string, // Array of at-uri strings + }, + }); + + // Pre-fill form from search params (from AI chat redirect) + useEffect(() \=\> { + if (params.id \=== 'new') { + const title \= searchParams.get('title') | + +| ''; + const body \= searchParams.get('body') | + +| ''; + form.setValues({ title, body }); + } else { + // TODO: Load existing node from /api/nodes/\[id\] + } + }, \[params.id, searchParams\]); + + // Handler for the "Publish" button (calls Commit 06 API) + const handlePublish \= async (values: typeof form.values) \=\> { + setIsPublishing(true); + try { + const response \= await fetch('/api/nodes', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify(values), + }); + + if (\!response.ok) { + throw new Error('Failed to publish node'); + } + + const newNode \= await response.json(); + // On success, go to the graph + router.push('/galaxy'); + + } catch (error) { + console.error(error); + // TODO: Show notification + } finally { + setIsPublishing(false); + } + }; + + // Handler for the "Find related" button + const handleFindRelated \= async () \=\> { + setIsFinding(true); + setSuggestions(); + try { + const response \= await fetch('/api/suggest-links', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ body: form.values.body }), + }); + + if (\!response.ok) { + throw new Error('Failed to find links'); + } + + const relatedNodes \= await response.json(); + setSuggestions(relatedNodes); + + } catch (error) { + console.error(error); + // TODO: Show notification + } finally { + setIsFinding(false); + } + }; + + return ( + \<Container size="md" py="xl"\> + \<form onSubmit={form.onSubmit(handlePublish)}\> + \<Stack gap="md"\> + \<Title order={2}\> + {params.id \=== 'new'? 'Create New Node' : 'Edit Node'} + \</Title\> + + \<TextInput + label="Title" + placeholder="Your node title" + required + {...form.getInputProps('title')} + /\> + + \<Textarea + label="Body" + placeholder="Your node content..." + required + minRows={10} + autosize + {...form.getInputProps('body')} + /\> + + \<Group\> + \<Button + type="button" + variant="outline" + onClick={handleFindRelated} + loading={isFinding} + \> + Find Related + \</Button\> + \<Button type="submit" loading={isPublishing}\> + Publish Node + \</Button\> + \</Group\> + + {/\* Related Links Section \*/} + \<Stack\> + {isFinding && \<LoadingOverlay visible /\>} + {suggestions.length \> 0 && \<Title order={4}\>Suggested Links\</Title\>} + + {suggestions.map((node) \=\> ( + \<Paper key={node.id} withBorder p="sm"\> + \<Text fw={700}\>{node.title}\</Text\> + \<Text size="sm" lineClamp={2}\>{node.body}\</Text\> + \<Text size="xs" c="dimmed"\>Similarity: {(node.score \* 100).toFixed(0)}%\</Text\> + \</Paper\> + ))} + + {\!isFinding && suggestions.length \=== 0 && ( + \<Text size="sm" c="dimmed"\> + {/\* Placeholder text \*/} + \</Text\> + )} + \</Stack\> + + \</Stack\> + \</form\> + \</Container\> + ); +} + +**2\. Create Link Suggestion API (app/api/suggest-links/route.ts)** + +Create a file at /app/api/suggest-links/route.ts: + +TypeScript + +import { NextRequest, NextResponse } from 'next/server'; +import { cookies } from 'next/headers'; +import { connectToDB } from '@/lib/db'; +import { generateEmbedding } from '@/lib/ai'; + +export async function POST(request: NextRequest) { + const surrealJwt \= cookies().get('ponderants-auth')?.value; + if (\!surrealJwt) { + return NextResponse.json({ error: 'Not authenticated' }, { status: 401 }); + } + + const { body } \= await request.json(); + if (\!body) { + return NextResponse.json({ error: 'Body text is required' }, { status: 400 }); + } + + try { + // 1\. Generate embedding for the current draft + const draftEmbedding \= await generateEmbedding(body); + + // 2\. Connect to DB (as the user) + const db \= await connectToDB(surrealJwt); + + // 3\. Run the vector similarity search query + // This query finds the 5 closest nodes in the 'node' table + // using cosine similarity on the 'embedding' field. + // It only searches nodes WHERE user\_did \= $token.did, + // which is enforced by the table's PERMISSIONS. + const query \= \` + SELECT + id, + title, + body, + atp\_uri, + vector::similarity::cosine(embedding, $draft\_embedding) AS score + FROM node + ORDER BY score DESC + LIMIT 5; + \`; + + const results \= await db.query(query, { + draft\_embedding: draftEmbedding, + }); + + // The query returns an array of result sets. We want the first one. + return NextResponse.json(results.result); + + } catch (error) { + console.error('Link suggestion error:', error); + return NextResponse.json( + { error: 'Failed to suggest links' }, + { status: 500 } + ); + } +} + +### **Test Specification** + +**1\. Create Test File (tests/magnitude/10-linking.mag.ts)** + +Create a file at /tests/magnitude/10-linking.mag.ts: + +TypeScript + +import { test } from 'magnitude-test'; + +// Helper function to seed the database for this test +async function seedDatabase(agent) { + // This would use a custom magnitude.run command or API + // to pre-populate the SurrealDB instance with mock nodes. + await agent.act('Seed the database with 3 nodes: "Node A", "Node B", "Node C"'); + // "Node A" is about "dogs and cats" + // "Node B" is about "vector databases" + // "Node C" is about "ATproto" +} + +test('\[Happy Path\] User can find related links for a draft', async (agent) \=\> { + // Setup: Seed the DB + await seedDatabase(agent); + + // Act: Navigate to the editor + await agent.act('Navigate to /editor/new'); + + // Act: Fill out the form with a related idea + await agent.act( + 'Enter "My New Post" into the "Title" input' + ); + await agent.act( + 'Enter "This idea is about vectors and databases, and how they work." into the "Body" textarea' + ); + + // Act: Click the find related button + // (Mock the /api/suggest-links route to return "Node B") + await agent.act('Click the "Find Related" button'); + + // Check: The related node appears in the suggestions + await agent.check('A list of suggested links appears'); + await agent.check('The suggested node "Node B" is visible in the list'); + await agent.check('The suggested node "Node A" is NOT visible in the list'); +}); + +test('\[Unhappy Path\] User sees empty state when no links found', async (agent) \=\> { + // Setup: Seed the DB + await seedDatabase(agent); + + // Act: Navigate to the editor + await agent.act('Navigate to /editor/new'); + + // Act: Fill out the form with an unrelated idea + await agent.act( + 'Enter "Zebras" into the "Title" input' + ); + await agent.act( + 'Enter "Zebras are striped equines." into the "Body" textarea' + ); + + // Act: Click the find related button + // (Mock the /api/suggest-links route to return an empty array) + await agent.act('Click the "Find Related" button'); + + // Check: An empty state is shown + await agent.check('The text "No related nodes found" is visible'); +}); + +test('\[Happy Path\] User can publish a new node', async (agent) \=\> { + // Act: Navigate to the editor + await agent.act('Navigate to /editor/new'); + + // Act: Fill out the form + await agent.act( + 'Enter "My First Published Node" into the "Title" input' + ); + await agent.act( + 'Enter "This is the body of my first node." into the "Body" textarea' + ); + + // Act: Click Publish + // (Mock the /api/nodes route (Commit 06\) to return success) + await agent.act('Click the "Publish Node" button'); + + // Check: User is redirected to the galaxy + await agent.check( + 'The browser URL is now "http://localhost:3000/galaxy"' + ); +});