dotfiles/README.md

# dotfiles

Centralized configuration and environment management for `kenjim`.  
Remote: `http://172.27.0.35:3000/kenjim/dotfiles`

---

## Overview

This repo tracks shell configs, SSH settings, and editor dotfiles via a git
repo hosted on a local Gitea server. On any machine, a single command restores
the full environment: directories, environment variables, shell aliases, and
symlinked config files.

Three scripts drive the system:

| Script | Role |
|---|---|
| `scripts/setup_enterprise_ai_bash.sh` | **First-time setup** — creates the directory tree, writes shell config, initializes dotfiles repo, migrates files |
| `scripts/dotfiles_manager.sh` | **Day-to-day management** — add/remove tracked files, push/pull, SSH key handling, multi-machine bootstrap |
| `scripts/bootstrap.sh` | **Restore shortcut** — reloads shell config and runs `dotfiles sync` to pull latest changes |
| `install.sh` | **New-machine restore** — applies all symlinks after cloning; auto-generated by `dotfiles_manager.sh` |

---

## Directory Layout

`setup_enterprise_ai_bash.sh` creates the following structure under `$HOME`:

```
~/
├── workspace/
│   ├── src/
│   │   ├── personal/          # personal projects
│   │   ├── work/              # work projects
│   │   └── research/          # research / papers
│   ├── experiments/           # throwaway experiments
│   ├── notebooks/             # Jupyter / Marimo notebooks
│   ├── sandboxes/             # isolated test environments
│   └── archive/               # completed / mothballed work
│
├── data/
│   ├── raw/                   # unmodified source data (excluded from Time Machine)
│   ├── processed/             # cleaned / transformed datasets
│   ├── embeddings/            # vector embeddings
│   └── synthetic/             # generated / synthetic data
│
├── models/                    # excluded from Time Machine
│   ├── huggingface/           # HF_HOME cache
│   ├── ollama/                # OLLAMA_MODELS store
│   └── fine-tuned/            # locally fine-tuned checkpoints
│
├── infra/
│   ├── docker/                # Compose files, Dockerfiles
│   ├── terraform/             # IaC configs
│   └── scripts/               # infra automation
│
├── ops/                       # operational runbooks, monitoring configs
├── scripts/                   # local utility scripts (dotfiles_manager, bootstrap, etc.)
├── vault/                     # secrets / credentials (never committed)
│
├── dotfiles/                  # ← this repo
│   ├── .bashrc                → symlinked from ~/.bashrc
│   ├── .bash_profile          → symlinked from ~/.bash_profile
│   ├── .bash_aliases          → symlinked from ~/.bash_aliases
│   ├── .gitconfig             → symlinked from ~/.gitconfig
│   ├── .vimrc                 → symlinked from ~/.vimrc
│   ├── .tmux.conf             → symlinked from ~/.tmux.conf
│   ├── .inputrc               → symlinked from ~/.inputrc
│   ├── .ssh/
│   │   ├── config             → symlinked from ~/.ssh/config
│   │   └── keys/              # .pub files + GPG-encrypted private keys
│   ├── scripts/
│   │   ├── dotfiles_manager.sh        → symlinked from ~/scripts/dotfiles_manager.sh
│   │   ├── setup_enterprise_ai_bash.sh → symlinked from ~/scripts/setup_enterprise_ai_bash.sh
│   │   └── bootstrap.sh               → symlinked from ~/scripts/bootstrap.sh
│   ├── .dotfiles_manifest     # internal list of tracked HOME-relative paths
│   ├── hosts/
│   │   └── <hostname>.bashrc.local  # per-server local configs (deployed via deploy-to)
│   ├── install.sh             # portable restore script (auto-generated)
│   └── README.md              # this file
│
└── Cloud/                     # personal profile only
    ├── ProtonDrive/           # PROTON_ROOT
    └── GoogleDrive/           # GDRIVE_ROOT
    # work profile instead creates:
    # ~/OneDrive → ~/Library/CloudStorage/OneDrive-*/  (symlink)
```

---

## Machine Profiles

The setup supports two profiles selected at first-run (or via `MACHINE_PROFILE=work|personal`):

| | `work` | `personal` |
|---|---|---|
| Cloud root | `~/OneDrive` (symlink to macOS mount) | `~/Cloud` |
| `CLOUD_ROOT` | `$HOME/OneDrive` | `$HOME/Cloud` |
| Extra vars | `ONEDRIVE_ROOT` | `PROTON_ROOT`, `GDRIVE_ROOT` |
| Extra aliases | `onedrive` | `cloud`, `proton`, `gdrive` |
| Committed? | No — written to `~/.bashrc.local` | No — written to `~/.bashrc.local` |

---

## Shell Config Layering

```
~/.bash_profile
  └── sources ~/.bashrc
        ├── ### ENTERPRISE_AI_ENV ###    ← shared vars (WORKSPACE, DATA_ROOT, MODEL_ROOT, …)
        ├── ### DOTFILES_ALIASES ###     ← dotfiles / dot / dots-sync / dots-push / dots-status
        └── sources ~/.bashrc.local      ← machine-specific (profile, cloud paths, MACHINE_HOST)
```

| File | Committed to dotfiles? | Purpose |
|---|---|---|
| `~/.bashrc` | **Yes** | Shared env vars, aliases, loads `.bashrc.local` |
| `~/.bash_profile` | **Yes** | macOS login shell — sources `.bashrc` |
| `~/.bashrc.local` | **No** | Machine-specific overrides (profile, cloud paths) |

`~/.bashrc.local` is written by `setup_enterprise_ai_bash.sh` and regenerated when
running it again. It is listed in `.gitignore` and never pushed to the remote.

---

## Key Environment Variables

| Variable | Value | Set in |
|---|---|---|
| `WORKSPACE` | `$HOME/workspace` | `~/.bashrc` |
| `DATA_ROOT` | `$HOME/data` | `~/.bashrc` |
| `MODEL_ROOT` | `$HOME/models` | `~/.bashrc` |
| `HF_HOME` | `$MODEL_ROOT/huggingface` | `~/.bashrc` |
| `OLLAMA_MODELS` | `$MODEL_ROOT/ollama` | `~/.bashrc` |
| `DOTFILES_DIR` | `$HOME/dotfiles` | `~/.bashrc` |
| `DOTFILES_REMOTE` | `http://172.27.0.35:3000/kenjim/dotfiles` | `~/.bashrc` |
| `MACHINE_PROFILE` | `work` or `personal` | `~/.bashrc.local` |
| `MACHINE_HOST` | `hostname -s` output | `~/.bashrc.local` |
| `CLOUD_ROOT` | `~/OneDrive` or `~/Cloud` | `~/.bashrc.local` |

---

## Shell Aliases

| Alias | Expands to |
|---|---|
| `ws` | `cd $WORKSPACE` |
| `src` | `cd $WORKSPACE/src` |
| `data` | `cd $DATA_ROOT` |
| `models` | `cd $MODEL_ROOT` |
| `cloud` | `cd $CLOUD_ROOT` |
| `dotfiles` / `dot` | `bash ~/scripts/dotfiles_manager.sh` |
| `dots-sync` | `dotfiles sync` |
| `dots-push` | `dotfiles push` |
| `dots-status` | `dotfiles status` |

---

## Bootstrap Flow

### First-time setup on a new machine

```bash
# 1. Run the full setup wizard (creates dirs, shell config, dotfiles repo):
bash scripts/setup_enterprise_ai_bash.sh

# 2. Reload shell:
source ~/.bash_profile
```

The wizard will interactively:
1. Select a machine profile (`work` / `personal`)
2. Create the full directory tree
3. Write `~/.bashrc` env block and `~/.bashrc.local`
4. Initialize or clone the dotfiles repo from Gitea
5. Add dotfiles aliases to `~/.bashrc`
6. Offer to migrate shell configs, `.gitconfig`, scripts into dotfiles
7. Offer SSH config setup + GPG key export
8. Offer to push everything to `$DOTFILES_REMOTE`
9. Offer Time Machine exclusions for `~/data/raw` and `~/models`
10. Offer to bootstrap a remote machine over SSH

### Restore an existing setup on a new machine

```bash
# Clone the repo and run the portable restore script:
git clone http://172.27.0.35:3000/kenjim/dotfiles ~/dotfiles
bash ~/dotfiles/install.sh
source ~/.bash_profile
```

`install.sh` will:
- Apply all symlinks listed in `.dotfiles_manifest`
- Prompt for machine profile and run `setup_enterprise_ai_bash.sh` if no `.bashrc.local` exists
- Prompt to GPG-decrypt any SSH private keys stored in `dotfiles/.ssh/keys/*.gpg`

### Day-to-day sync

After editing on one machine, push and sync on the other:

```bash
# On the machine where you made changes:
dotfiles push "describe what changed"

# On the other machine to pick up the changes:
dotfiles sync
```

```bash
# Other useful commands:
dotfiles status        # check symlink health
dotfiles sync          # pull + rebase + reapply symlinks (also callable as: bash ~/scripts/bootstrap.sh)
```

### Remote machine bootstrap (from this machine)

```bash
dotfiles remote-bootstrap user@hostname
dotfiles remote-bootstrap user@hostname --profile work
```

This uploads the scripts, then runs the full setup interactively over SSH.

### Deploy to a server with no Gitea access

Linux servers at work can't clone the private Gitea repo. Use `deploy-to` to
SCP tracked dotfiles and scripts directly to their HOME paths — no git required
on the remote.

```bash
# Push everything (dotfiles + scripts):
dotfiles deploy-to user@server

# After file transfer, also run setup_enterprise_ai_bash.sh on the remote
# to create ~/workspace, ~/data, ~/models and write shell config:
dotfiles deploy-to user@server --run-setup

# Specify a machine profile for the setup script (default: work):
dotfiles deploy-to user@server --run-setup --profile work

# Scripts only (dotfiles_manager.sh, bootstrap.sh, setup script):
dotfiles deploy-to user@server --scripts-only

# Preview what would be transferred without doing it:
dotfiles deploy-to user@server --dry-run

# Also deploy ~/.ssh/config (skipped by default for security):
dotfiles deploy-to user@server --include-ssh

# Skip the pre-deploy backup (faster, but no safety net):
dotfiles deploy-to user@server --no-backup
```

Before overwriting any file, `deploy-to` SCPs the server's existing versions
to `~/.dotfiles_backup/remote-<host>-<timestamp>/` on your local machine.
Files are copied directly (not symlinked). Re-run `deploy-to` any time you
want to push updates. `~/.ssh/` is skipped by default to avoid accidentally
pushing private keys or your personal known_hosts to a shared server.

> **First-time setup on a server?** Use `--run-setup` to create the full
> directory structure after file transfer, or answer `y` when prompted
> interactively at the end of a normal `deploy-to` run.

### Centrally managing `~/.bashrc.local` for servers

Work servers can't reach the Gitea repo, so their `~/.bashrc.local` is managed
centrally from `kenjim-mbp` using per-host files in `dotfiles/hosts/`:

```
dotfiles/hosts/
└── <hostname>.bashrc.local    # deployed as ~/.bashrc.local on that server
```

`deploy-to` automatically detects and deploys the matching file:

```bash
# Edit the server's local config on kenjim-mbp:
$EDITOR ~/dotfiles/hosts/etqc-kenjim-11.bashrc.local

# Commit and push from kenjim-mbp:
dotfiles push "fix: update etqc-kenjim-11 local config"

# Deploy to the server (no git access needed on the server):
dotfiles deploy-to kenjim@etqc-kenjim-11
```

The `hosts/` files are committed to git. They may contain non-secret
machine-specific variables (`MACHINE_PROFILE`, `MACHINE_HOST`, `AWS_PROFILE`,
etc.). **Do not commit real passwords or tokens** — use `CHANGEME` placeholders
and set real values manually on the server after first deploy.

---

## Dotfiles Management — How Symlinks Work

```
~/dotfiles/.bashrc   ←──────────── ~/dotfiles/ (git repo)
      ↑
  symlink
      │
~/.bashrc            ←── shell reads this
```

`dotfiles add <file>` moves the file into `~/dotfiles/`, creates a symlink at
the original HOME location, and records the relative path in `.dotfiles_manifest`.
`dotfiles install` re-creates all symlinks from the manifest (idempotent — safe to
re-run). Conflicts are backed up to `~/.dotfiles_backup/<timestamp>/` before being
replaced.

---

## SSH Key Strategy

| Item | Committed? | Details |
|---|---|---|
| `~/.ssh/config` | **Yes** | Shared SSH client configuration |
| `~/.ssh/*.pub` | **Yes** | Public keys are safe to store |
| `~/.ssh/id_*` (private) | **No** | Listed in `.gitignore` |
| `~/.ssh/keys/*.gpg` | **Yes** | GPG-encrypted private key backups |

```bash
# Encrypt private keys and save to dotfiles:
dotfiles ssh-export

# Decrypt on another machine after sync:
dotfiles ssh-import
```

---

## Gitea Authentication

HTTP push/pull uses a Personal Access Token saved locally (never committed):

```bash
dotfiles auth
# → Settings (avatar) → Applications → Generate Token → paste here
```

Credentials are stored at `~/.config/dotfiles/credentials` (chmod 600).

---

## Time Machine Exclusions

During setup you are offered the option to exclude large AI folders:

```
~/data/raw      — large raw datasets
~/models        — multi-GB model weights
```

To add manually:
```bash
sudo tmutil addexclusion ~/data/raw
sudo tmutil addexclusion ~/models
```

**Ollama Bootstrap & Usage**

- **Bootstrap (macOS)**: Run the macOS installer/check + create model folders:

    ```bash
    bash scripts/bootstrap_ollama_mac.sh
    ```

- **Bootstrap (Linux)**: Run the Linux helper (attempts to detect package manager and prepares model folders):

    ```bash
    bash scripts/bootstrap_ollama_linux.sh
    ```

- The scripts create the model directory referenced by the setup script as `OLLAMA_MODELS` (by default `$HOME/models/ollama`).

- Start the Ollama server inside a tmux session (helper functions):

    ```bash
    # Load helper functions (one-liner to source from dotfiles workspace)
    source scripts/ollama_tmux.sh

    # Start the server in a detached tmux session named 'ollama' (default command):
    start_ollama_tmux

    # To attach to the session:
    tmux attach -t ollama

    # To stop the server session:
    stop_ollama_tmux
    ```

- Model management (two simple options):

    - Pull a remote model (via the `ollama` CLI):

        ```bash
        # example: pull a model by reference
        pull_ollama_model <model-ref>
        ```

    - Import or stage a local model into the $OLLAMA_MODELS path (creates a symlink):

        ```bash
        import_local_model /path/to/local/model-dir-or-file
        # The helper will link it into the directory created by setup_enterprise_ai_bash.sh
        ```

- Scripts referenced:

    - [scripts/bootstrap_ollama_mac.sh](scripts/bootstrap_ollama_mac.sh)
    - [scripts/bootstrap_ollama_linux.sh](scripts/bootstrap_ollama_linux.sh)
    - [scripts/ollama_tmux.sh](scripts/ollama_tmux.sh)

- How to use Ollama from other AI applications:

    - Run the server with `start_ollama_tmux` (above). By default the helper uses `ollama serve --host 127.0.0.1 --port 11434`.
    - Many client apps can be pointed at the local Ollama HTTP API (default: `http://127.0.0.1:11434`). Check the Ollama docs for exact endpoints and payloads.
    - Example using the `ollama` CLI to run a model directly from scripts or other programs:

        ```bash
        # Run a model locally (interactive / CLI). Replace <model> with the model name.
        ollama run <model> --prompt "Write a short haiku about trees"
        ```

    - Example (generic) using HTTP from another application (replace endpoint/payload per Ollama docs):

        ```bash
        curl -X POST "http://127.0.0.1:11434/api/generate" \
            -H "Content-Type: application/json" \
            -d '{"model":"<model>","prompt":"Hello from my app"}'
        ```

    - When integrating from other AI tooling, supply the Ollama host/port (e.g. `OLLAMA_URL=http://127.0.0.1:11434`) or call the `ollama` CLI directly from the application process.

If you want, I can also:

- Make the new scripts executable (`chmod +x`) automatically in the repo
- Attempt to detect the Ollama default model storage path and auto-configure a symlink