feat: Improve UI layout and navigation
- Increase logo size (48x48 desktop, 56x56 mobile) for better visibility - Add logo as favicon - Add logo to mobile header - Move user menu to navigation bars (sidebar on desktop, bottom bar on mobile) - Fix desktop chat layout - container structure prevents voice controls cutoff - Fix mobile bottom bar - use icon-only ActionIcons instead of truncated text buttons - Hide Create Node/New Conversation buttons on mobile to save header space - Make fixed header and voice controls work properly with containers 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
349
docs/fixes/galaxy-graph-fix.md
Normal file
349
docs/fixes/galaxy-graph-fix.md
Normal file
@@ -0,0 +1,349 @@
|
||||
# Galaxy Graph Visualization Fix
|
||||
|
||||
## Problems
|
||||
|
||||
1. **Invalid URL Error**: ThoughtGalaxy component was failing with:
|
||||
```
|
||||
TypeError: Failed to construct 'URL': Invalid URL
|
||||
at parseUrl (surreal.ts:745:14)
|
||||
at Surreal.connectInner (surreal.ts:93:20)
|
||||
at Surreal.connect (surreal.ts:84:22)
|
||||
at fetchData (ThoughtGalaxy.tsx:76:16)
|
||||
```
|
||||
|
||||
2. **Manual Calculation Required**: Users had to manually click "Calculate My Graph" button to trigger UMAP dimensionality reduction
|
||||
|
||||
3. **"Not enough nodes" despite having 3+ nodes**: System was reporting insufficient nodes even after creating 3+ nodes with content
|
||||
|
||||
## Root Causes
|
||||
|
||||
### 1. Client-Side Database Connection
|
||||
The `ThoughtGalaxy.tsx` client component was attempting to connect directly to SurrealDB:
|
||||
|
||||
```typescript
|
||||
// ❌ Wrong: Client component trying to connect to database
|
||||
import Surreal from 'surrealdb';
|
||||
|
||||
useEffect(() => {
|
||||
const db = new Surreal();
|
||||
await db.connect(process.env.NEXT_PUBLIC_SURREALDB_WSS_URL!); // undefined!
|
||||
// ...
|
||||
}, []);
|
||||
```
|
||||
|
||||
Problems:
|
||||
- `NEXT_PUBLIC_SURREALDB_WSS_URL` environment variable didn't exist
|
||||
- Client components shouldn't connect directly to databases (security/architecture violation)
|
||||
- No authentication handling on client side
|
||||
|
||||
### 2. Manual Trigger Required
|
||||
Graph calculation only happened when user clicked a button. No automatic detection of when calculation was needed.
|
||||
|
||||
### 3. Connection Method Inconsistency
|
||||
The `calculate-graph` route was using inline database connection instead of the shared `connectToDB()` helper, leading to potential authentication mismatches.
|
||||
|
||||
## Solutions
|
||||
|
||||
### 1. Created Server-Side Galaxy API Route
|
||||
Created `/app/api/galaxy/route.ts` to handle all database access server-side:
|
||||
|
||||
```typescript
|
||||
export async function GET(request: NextRequest) {
|
||||
const cookieStore = await cookies();
|
||||
const surrealJwt = cookieStore.get('ponderants-auth')?.value;
|
||||
|
||||
if (!surrealJwt) {
|
||||
return NextResponse.json({ error: 'Not authenticated' }, { status: 401 });
|
||||
}
|
||||
|
||||
const userSession = verifySurrealJwt(surrealJwt);
|
||||
if (!userSession) {
|
||||
return NextResponse.json({ error: 'Invalid auth token' }, { status: 401 });
|
||||
}
|
||||
|
||||
const { did: userDid } = userSession;
|
||||
|
||||
try {
|
||||
const db = await connectToDB();
|
||||
|
||||
// Fetch nodes that have 3D coordinates
|
||||
const nodesQuery = `
|
||||
SELECT id, title, coords_3d
|
||||
FROM node
|
||||
WHERE user_did = $userDid AND coords_3d != NONE
|
||||
`;
|
||||
const nodeResults = await db.query<[NodeData[]]>(nodesQuery, { userDid });
|
||||
const nodes = nodeResults[0] || [];
|
||||
|
||||
// Fetch links between nodes
|
||||
const linksQuery = `
|
||||
SELECT in, out
|
||||
FROM links_to
|
||||
`;
|
||||
const linkResults = await db.query<[LinkData[]]>(linksQuery);
|
||||
const links = linkResults[0] || [];
|
||||
|
||||
// Auto-trigger calculation if needed
|
||||
if (nodes.length === 0) {
|
||||
const unmappedQuery = `
|
||||
SELECT count() as count
|
||||
FROM node
|
||||
WHERE user_did = $userDid AND embedding != NONE AND coords_3d = NONE
|
||||
GROUP ALL
|
||||
`;
|
||||
const unmappedResults = await db.query<[Array<{ count: number }>]>(unmappedQuery, { userDid });
|
||||
const unmappedCount = unmappedResults[0]?.[0]?.count || 0;
|
||||
|
||||
if (unmappedCount >= 3) {
|
||||
console.log(`[Galaxy API] Found ${unmappedCount} unmapped nodes, triggering calculation...`);
|
||||
|
||||
// Trigger graph calculation (don't await, return current state)
|
||||
fetch(`${process.env.NEXT_PUBLIC_BASE_URL || 'http://localhost:3000'}/api/calculate-graph`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Cookie': `ponderants-auth=${surrealJwt}`,
|
||||
},
|
||||
}).catch((err) => {
|
||||
console.error('[Galaxy API] Failed to trigger graph calculation:', err);
|
||||
});
|
||||
|
||||
return NextResponse.json({
|
||||
nodes: [],
|
||||
links: [],
|
||||
message: 'Calculating 3D coordinates... Refresh in a moment.',
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
console.log(`[Galaxy API] Returning ${nodes.length} nodes and ${links.length} links`);
|
||||
|
||||
return NextResponse.json({
|
||||
nodes,
|
||||
links,
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('[Galaxy API] Error:', error);
|
||||
return NextResponse.json(
|
||||
{ error: 'Failed to fetch galaxy data' },
|
||||
{ status: 500 }
|
||||
);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Key features:
|
||||
- ✅ Server-side authentication with JWT verification
|
||||
- ✅ Data isolation via `user_did` filtering
|
||||
- ✅ Auto-detection of unmapped nodes
|
||||
- ✅ Automatic triggering of UMAP calculation
|
||||
- ✅ Progress messaging for client polling
|
||||
|
||||
### 2. Updated ThoughtGalaxy Component
|
||||
Changed from direct database connection to API-based data fetching:
|
||||
|
||||
**Before:**
|
||||
```typescript
|
||||
import Surreal from 'surrealdb';
|
||||
|
||||
useEffect(() => {
|
||||
async function fetchData() {
|
||||
const db = new Surreal();
|
||||
await db.connect(process.env.NEXT_PUBLIC_SURREALDB_WSS_URL!);
|
||||
const token = document.cookie.split('ponderants-auth=')[1];
|
||||
await db.authenticate(token);
|
||||
|
||||
const nodeResults = await db.query('SELECT id, title, coords_3d FROM node...');
|
||||
setNodes(nodeResults[0] || []);
|
||||
}
|
||||
fetchData();
|
||||
}, []);
|
||||
```
|
||||
|
||||
**After:**
|
||||
```typescript
|
||||
// No Surreal import needed
|
||||
|
||||
useEffect(() => {
|
||||
async function fetchData() {
|
||||
try {
|
||||
const response = await fetch('/api/galaxy', {
|
||||
credentials: 'include', // Include cookies for authentication
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
console.error('[ThoughtGalaxy] Failed to fetch galaxy data:', response.statusText);
|
||||
return;
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
if (data.message) {
|
||||
console.log('[ThoughtGalaxy]', data.message);
|
||||
// If calculating, poll again in 2 seconds
|
||||
setTimeout(fetchData, 2000);
|
||||
return;
|
||||
}
|
||||
|
||||
setNodes(data.nodes || []);
|
||||
setLinks(data.links || []);
|
||||
|
||||
console.log(`[ThoughtGalaxy] Loaded ${data.nodes?.length || 0} nodes and ${data.links?.length || 0} links`);
|
||||
} catch (error) {
|
||||
console.error('[ThoughtGalaxy] Error fetching data:', error);
|
||||
}
|
||||
}
|
||||
|
||||
fetchData();
|
||||
}, []);
|
||||
```
|
||||
|
||||
Key improvements:
|
||||
- ✅ No client-side database connection
|
||||
- ✅ Proper authentication via HTTP-only cookies
|
||||
- ✅ Polling mechanism for in-progress calculations
|
||||
- ✅ Better error handling
|
||||
|
||||
### 3. Fixed calculate-graph Route
|
||||
Updated `/app/api/calculate-graph/route.ts` to use shared helpers:
|
||||
|
||||
**Before:**
|
||||
```typescript
|
||||
const db = new (await import('surrealdb')).default();
|
||||
await db.connect(process.env.SURREALDB_URL!);
|
||||
await db.signin({
|
||||
username: process.env.SURREALDB_USER!,
|
||||
password: process.env.SURREALDB_PASS!,
|
||||
});
|
||||
|
||||
const jwt = require('jsonwebtoken');
|
||||
const decoded = jwt.decode(surrealJwt) as { did: string };
|
||||
const userDid = decoded?.did;
|
||||
```
|
||||
|
||||
**After:**
|
||||
```typescript
|
||||
import { connectToDB } from '@/lib/db';
|
||||
import { verifySurrealJwt } from '@/lib/auth/jwt';
|
||||
|
||||
const userSession = verifySurrealJwt(surrealJwt);
|
||||
if (!userSession) {
|
||||
return NextResponse.json({ error: 'Invalid auth token' }, { status: 401 });
|
||||
}
|
||||
|
||||
const { did: userDid } = userSession;
|
||||
|
||||
const db = await connectToDB();
|
||||
```
|
||||
|
||||
Benefits:
|
||||
- ✅ Consistent authentication across all routes
|
||||
- ✅ Proper JWT verification (not just decode)
|
||||
- ✅ Reusable code (DRY principle)
|
||||
|
||||
### 4. Created Debug Endpoint
|
||||
Added `/app/api/debug/nodes/route.ts` for database inspection:
|
||||
|
||||
```typescript
|
||||
export async function GET(request: NextRequest) {
|
||||
const cookieStore = await cookies();
|
||||
const surrealJwt = cookieStore.get('ponderants-auth')?.value;
|
||||
|
||||
if (!surrealJwt) {
|
||||
return NextResponse.json({ error: 'Not authenticated' }, { status: 401 });
|
||||
}
|
||||
|
||||
const userSession = verifySurrealJwt(surrealJwt);
|
||||
if (!userSession) {
|
||||
return NextResponse.json({ error: 'Invalid auth token' }, { status: 401 });
|
||||
}
|
||||
|
||||
const { did: userDid } = userSession;
|
||||
|
||||
try {
|
||||
const db = await connectToDB();
|
||||
|
||||
const nodesQuery = `
|
||||
SELECT id, title, body, atp_uri, embedding, coords_3d
|
||||
FROM node
|
||||
WHERE user_did = $userDid
|
||||
`;
|
||||
const results = await db.query(nodesQuery, { userDid });
|
||||
const nodes = results[0] || [];
|
||||
|
||||
const stats = {
|
||||
total: nodes.length,
|
||||
with_embeddings: nodes.filter((n: any) => n.embedding).length,
|
||||
with_coords: nodes.filter((n: any) => n.coords_3d).length,
|
||||
without_embeddings: nodes.filter((n: any) => !n.embedding).length,
|
||||
without_coords: nodes.filter((n: any) => !n.coords_3d).length,
|
||||
};
|
||||
|
||||
return NextResponse.json({
|
||||
stats,
|
||||
nodes: nodes.map((n: any) => ({
|
||||
id: n.id,
|
||||
title: n.title,
|
||||
atp_uri: n.atp_uri,
|
||||
has_embedding: !!n.embedding,
|
||||
has_coords: !!n.coords_3d,
|
||||
coords_3d: n.coords_3d,
|
||||
})),
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('[Debug Nodes] Error:', error);
|
||||
return NextResponse.json({ error: String(error) }, { status: 500 });
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Use: Visit `/api/debug/nodes` while logged in to see your node statistics and data.
|
||||
|
||||
## Auto-Calculation Flow
|
||||
|
||||
1. **User visits Galaxy page** → ThoughtGalaxy component mounts
|
||||
2. **Component fetches data** → `GET /api/galaxy`
|
||||
3. **API checks for coords** → Query: `WHERE coords_3d != NONE`
|
||||
4. **If no coords found** → Query unmapped count: `WHERE embedding != NONE AND coords_3d = NONE`
|
||||
5. **If ≥3 unmapped nodes** → Trigger `POST /api/calculate-graph` (don't wait)
|
||||
6. **Return progress message** → `{ message: 'Calculating 3D coordinates...' }`
|
||||
7. **Client polls** → setTimeout 2 seconds, fetch again
|
||||
8. **UMAP completes** → Next poll returns actual node data
|
||||
9. **Client renders** → 3D visualization appears
|
||||
|
||||
## Files Changed
|
||||
|
||||
1. `/components/ThoughtGalaxy.tsx` - Removed direct DB connection, added API-based fetching and polling
|
||||
2. `/app/api/galaxy/route.ts` - **NEW** - Server-side galaxy data endpoint with auto-calculation
|
||||
3. `/app/api/calculate-graph/route.ts` - Updated to use `connectToDB()` and `verifySurrealJwt()`
|
||||
4. `/app/api/debug/nodes/route.ts` - **NEW** - Debug endpoint for inspecting node data
|
||||
|
||||
## Verification
|
||||
|
||||
After the fix:
|
||||
|
||||
```bash
|
||||
# Server logs show auto-calculation:
|
||||
[Galaxy API] Found 5 unmapped nodes, triggering calculation...
|
||||
[Calculate Graph] Processing 5 nodes for UMAP projection
|
||||
[Calculate Graph] Running UMAP dimensionality reduction...
|
||||
[Calculate Graph] ✓ UMAP projection complete
|
||||
[Calculate Graph] ✓ Updated 5 nodes with 3D coordinates
|
||||
```
|
||||
|
||||
```bash
|
||||
# Client logs show polling:
|
||||
[ThoughtGalaxy] Calculating 3D coordinates... Refresh in a moment.
|
||||
[ThoughtGalaxy] Loaded 5 nodes and 3 links
|
||||
```
|
||||
|
||||
## Architecture Note
|
||||
|
||||
This fix maintains the "Source of Truth vs. App View Cache" pattern:
|
||||
- **ATproto PDS** - Canonical source of Node content (com.ponderants.node records)
|
||||
- **SurrealDB** - Performance cache that stores:
|
||||
- Copy of node data for fast access
|
||||
- Vector embeddings for similarity search
|
||||
- Pre-computed 3D coordinates for visualization
|
||||
- Graph links between nodes
|
||||
|
||||
The auto-calculation ensures the cache stays enriched with visualization data without user intervention.
|
||||
Reference in New Issue
Block a user