Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,7 @@ data.json
# Exclude files specific to personal development process
.hotreload
.devtarget

# Local docs
_DOCS
_gpt5_docs
71 changes: 71 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

Obsidian Importer is an Obsidian plugin that imports notes from various formats (Notion, Evernote, Apple Notes, OneNote, Google Keep, Bear, Roam, CSV, HTML, etc.) into Obsidian vaults.

## Commands

- **Build**: `npm run build` (runs `tsc -skipLibCheck` then esbuild production bundle)
- **Dev**: `npm run dev` (esbuild watch mode with inline sourcemaps)
- **Lint**: `npm run lint` (eslint with `--fix`, uses flat config)

There is no automated test suite. The `tests/` directory contains fixture data for manual testing. Verify changes by building and testing the plugin in Obsidian.

## Architecture

### Plugin Entry Point

`src/main.ts` — defines `ImporterPlugin` (extends Obsidian `Plugin`), `ImporterModal` (the UI), and `ImportContext` (progress tracking during imports). All importers are registered in the `importers` map inside `ImporterPlugin`.

### Importer Pattern

Every importer extends `FormatImporter` (`src/format-importer.ts`):

1. **`init()`** — adds UI settings to `this.modal.contentEl` (file chooser, output location, format-specific options)
2. **`showTemplateConfiguration(ctx, container)`** — optional second config screen (e.g., CSV column mapping). Returns `true`/`false`/`null`.
3. **`import(ctx: ImportContext)`** — performs the actual import. Report progress via `ctx.reportNoteSuccess()`, `ctx.reportAttachmentSuccess()`, `ctx.reportSkipped()`, `ctx.reportFailed()`, `ctx.reportProgress()`.

Format implementations live in `src/formats/` — each is a file or directory.

### Registering a New Importer

Add entry to the `importers` object in `src/main.ts`:
```ts
'format-id': {
name: 'Display Name',
optionText: 'Dropdown text (.ext)',
helpPermalink: 'import/format-id',
importer: YourImporterClass,
}
```

### Key Modules

- `src/filesystem.ts` — file I/O abstraction (`PickedFile`, `NodePickedFile`, `WebPickedFile`). Must use this instead of direct Node.js `fs`/`path` imports.
- `src/template.ts` — template system for structured data (CSV, etc.). Substitutes `{{fieldName}}` placeholders and generates YAML frontmatter.
- `src/util.ts` — filename sanitization, text utilities.
- `src/zip.ts` — zip file handling via `@zip.js/zip.js`.

### Cross-Platform Compatibility

The plugin runs on desktop (Electron/Node.js) and mobile (web). Node.js modules must be soft-imported:
```ts
import type * as NodeModuleName from 'node:modulename';
const modulename: typeof NodeModuleName = Platform.isDesktopApp ? window.require('node:modulename') : null;
```

Some importers (e.g., Apple Notes) are desktop-only and set `this.notAvailable = true` on unsupported platforms.

## Code Style

- TypeScript only, strict mode
- Tabs for indentation, single quotes, semicolons
- Stroustrup brace style
- Unused function args are allowed (prefixed with `_` or not)
- `any` type is permitted
- Minimal dependencies — avoid heavy libraries
- Avoid concurrency (sequential processing to prevent memory issues with large vaults)
- esbuild bundles to single `main.js`; `obsidian`, `electron`, and `codemirror` are external
81 changes: 73 additions & 8 deletions src/formats/roam-json.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ const imageRegex = /https:\/\/firebasestorage(.*?)\?alt(.*?)\)/;
const binaryRegex = /https:\/\/firebasestorage(.*?)\?alt(.*?)/;

const blockRefRegex = /(?<=\(\()\b(.*?)\b(?=\)\))/g;
const roamTableRe = /^\{\{(\[\[)?table(\]\])?\}\}$/i;
const codeBlockLangs = ['clojure', 'css', 'elixir', 'html', 'plain text', 'python', 'ruby', 'swift', 'typescript', 'jsx', 'yaml', 'json', 'json-ld', 'rust', 'r', 'shell', 'php', 'java', 'c#', 'c', 'c\\+\\+', 'objective-c', 'go', 'kotlin', 'sql', 'haskell', 'scala', 'commonlisp', 'solidity', 'julia', 'sparql', 'turtle', 'lua', 'dart', 'latex', 'markdown', 'xml', 'toml', 'vb', 'vbscript', 'javascript'];

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused why this is needed. Why are these tags being removed?

const codeBlockLangRe = new RegExp('```(' + codeBlockLangs.join('|') + ')\\n', 'gi');

export class RoamJSONImporter extends FormatImporter {
downloadAttachments: boolean = false;
Expand Down Expand Up @@ -248,6 +251,13 @@ export class RoamJSONImporter extends FormatImporter {
}

private async roamMarkupScrubber(graphFolder: string, attachmentsFolder: string, blockText: string, skipDownload: boolean = false): Promise<string> {
// Strip language tags from code blocks (```javascript\n → ```\n)
blockText = blockText.replace(codeBlockLangRe, '```\n');

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are these being stripped? Obsidian supports these.

// Normalize code fences: ensure closing ``` is on its own line
blockText = blockText.replace(/([^\n])```/g, '$1\n```');
// Ensure opening ``` followed by content has a newline after it
blockText = blockText.replace(/```([^\n])/g, '```\n$1');

// Remove roam-specific components
blockText = blockText.replace(roamSpecificMarkupRe, '');

Expand Down Expand Up @@ -286,7 +296,7 @@ export class RoamJSONImporter extends FormatImporter {
blockText = await this.downloadFirebaseFile(blockText, attachmentsFolder);
}
}
// blockText = blockText.replaceAll("{{[[table]]}}", "");
// table conversion is handled in jsonToMarkdown via convertRoamTable
// blockText = blockText.replaceAll("{{[[kanban]]}}", "");
// blockText = blockText.replaceAll("{{mermaid}}", "");
// blockText = blockText.replaceAll("{{[[mermaid]]}}", "");
Expand Down Expand Up @@ -339,15 +349,25 @@ export class RoamJSONImporter extends FormatImporter {
this.oldestTimestamp = createdTimestamp;
}

if ('string' in json && json.string) {
const prefix = json.heading ? '#'.repeat(json.heading) + ' ' : '';
const scrubbed = await this.roamMarkupScrubber(graphFolder, attachmentsFolder, json.string);
markdown.push(`${isChild ? indent + '* ' : indent}${prefix}${scrubbed}`);
if ('string' in json && json.string && roamTableRe.test(json.string.trim()) && json.children) {
markdown.push(await this.convertRoamTable(graphFolder, attachmentsFolder, json, indent));
}
else {
if ('string' in json && json.string) {
const prefix = json.heading ? '#'.repeat(json.heading) + ' ' : '';
const scrubbed = await this.roamMarkupScrubber(graphFolder, attachmentsFolder, json.string);
const linePrefix = isChild ? indent + '* ' : indent;
const continuationIndent = isChild ? indent + ' ' : indent;
const indented = scrubbed.contains('\n')
? scrubbed.split('\n').map((line, i) => i === 0 ? line : continuationIndent + line).join('\n')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should avoid prepending whitespace to blank lines. You also don't need to the check for newlines.

const indented = scrubbed
    .split('\n')
    .map((line, i) => (i === 0 || line === '') ? line : continuationIndent + line)
    .join('\n');

: scrubbed;
markdown.push(`${linePrefix}${prefix}${indented}`);
}

if (json.children) {
for (const child of json.children) {
markdown.push(await this.jsonToMarkdown(graphFolder, attachmentsFolder, child, indent + ' ', true, '', this.oldestTimestamp, this.newestTimestamp));
if (json.children) {
for (const child of json.children) {
markdown.push(await this.jsonToMarkdown(graphFolder, attachmentsFolder, child, indent + ' ', true, '', this.oldestTimestamp, this.newestTimestamp));
}
}
}

Expand Down Expand Up @@ -387,6 +407,51 @@ export class RoamJSONImporter extends FormatImporter {
return markdown.join('\n');
}

private async convertRoamTable(
graphFolder: string,
attachmentsFolder: string,
json: RoamPage | RoamBlock,
indent: string
): Promise<string> {
const rows = json.children || [];
if (rows.length === 0) return '';

// Extract cells from each row by walking the linear first-child chain
const tableData: string[][] = [];
for (const row of rows) {
const cells: string[] = [];
let current: RoamBlock | undefined = row;
while (current) {
const scrubbed = await this.roamMarkupScrubber(graphFolder, attachmentsFolder, current.string || '');
cells.push(scrubbed.replace(/\|/g, '\\|'));
current = current.children?.[0];
}
tableData.push(cells);
}

// Determine max columns and pad shorter rows
const numCols = Math.max(...tableData.map(r => r.length));
for (const row of tableData) {
while (row.length < numCols) {
row.push('');
}
}

// Build pipe table
const lines: string[] = [];
// Header row
lines.push(indent + '| ' + tableData[0].join(' | ') + ' |');
// Separator
lines.push(indent + '| ' + tableData[0].map(() => '---').join(' | ') + ' |');
// Data rows
for (let i = 1; i < tableData.length; i++) {
lines.push(indent + '| ' + tableData[i].join(' | ') + ' |');
}

// Blank line before table for clean rendering
return '\n' + lines.join('\n');
}

private async modifySourceBlockString(markdownPages: Map<string, string>, sourceBlock: BlockInfo, graphFolder: string, sourceBlockUID: string) {
if (!sourceBlock.blockString.endsWith('^' + sourceBlockUID)) {
const sourceBlockFilePath = `${graphFolder}/${sourceBlock.pageName}.md`;
Expand Down