Languages
Python, TypeScript, JavaScript, Go
AI safety research and agentic systems
Aaliyan Shaikh
Software engineer and independent AI safety researcher working on evaluation reliability and agent systems
I build production AI systems and study how to evaluate them reliably. My research covers post-training behavior, mechanistic interpretability, and the failure modes of tool-using agents. I also mentor research through SPAR.
Capabilities
Stack and system instincts
AI and ML
PyTorch, Transformers, Hugging Face, LoRA/QLoRA, Gemini, evaluation harnesses and loops
AI Safety
Mechanistic interpretability, activation analysis, steering vectors, RLHF, red-teaming, safety evaluations
Backend
Django, DRF, Channels, Celery, Django-Q, Node.js, NestJS, retries, failure handling
Cloud and Ops
AWS, Docker, Terraform, Cloudflare, Linux, CI, observability tooling, production hardening, latency and cost tradeoffs
Frontend
React, Vite, Next.js, Redux Toolkit, Tailwind CSS, shadcn/ui
Experience
Roles and responsibilities
Research Mentor
SPAR (Kairos)
- Mentoring the Fall 2026 project “When RLVR Changes the Model, the Safety Test, or Both,” building on the completed Tülu checkpoint study.
- Guiding experiment design, evaluation protocols, and research implementation for the continuation.
Senior AI Software Engineer
OCN
- Built a production agentic platform across Django and ASGI services, React frontends, relational data flows, and async job execution.
- Shipped chat, voice, retrieval, and document-processing workflows with a strong focus on tool reliability, concurrency, and operational stability.
- Supported cloud deployment and hardening across AWS infrastructure, worker orchestration, key management, and real-time customer workflows.
Software Engineer
Sastaticket
- Built product features across backend and frontend systems for core web workflows.
- Contributed AI-focused prototypes to evaluate NLP use cases for the platform.
Software Engineer
Techwards
- Built and maintained AI-enabled web applications across backend and frontend services, shipping interactive product experiences while improving legacy frontend quality.
- Implemented backend capabilities for retrieval workflows, access control, internal request management, and enterprise chatbot flows with deeper analysis and tool-assisted interactions.
- Delivered AI-assisted automation features for publishing, research, and internal product use cases.
Projects
Systems, benchmarks, and tools
Agent Watchtower
Agent Watchtower
A flight recorder and verifier for AI agent runs. Python records model and tool calls as OpenTelemetry spans; Go reconstructs runs and checks them deterministically. Every verdict includes evidence, and incomplete traces produce an explicit INCONCLUSIVE result.
Tool Calling Reliability Benchmark
Tool Calling Reliability Benchmark
Fault-injected benchmark for tool-using agents, with targeted planner repairs, transfer checks, and evidence-based monitoring pilots. Evaluates routing, recovery, and unsafe pre-write actions in stateful workflows.
OCN Agents (OneCarNow)
OCN Agents (OneCarNow)
Production agentic platform for chat, voice, and KYC workflows across Django services, async workers, retrieval and tool execution, and React delivery.
CanaryGame
CanaryGame
A synthetic multi-agent security testbed for shared-memory honeytokens. Compares decoy rotation with brokered containment, isolated trap namespaces, and quarantine while adaptive agents share information.
Research and Writing
Investigations and long-form thinking
Research
SPAR Research Mentor, Fall 2026 Poster @ TAE, NeurIPS 2026
When the Judge Drifts: Separating Safety Behavior from Its Measurement Along a GRPO Trajectory
A checkpoint-level audit of safety measurement during RLVR, following Tülu 3.1 8B across 12 checkpoints and 6,912 structured responses. The frozen tests found a late increase in answer-order sensitivity without detecting a persistent safety shift that met the study's threshold. The study covers one training lineage; a SPAR continuation is in progress.
Non-archival workshop paper. Developed through the BlueDot Technical AI Safety project.
Poster @ SLM-Agents, NeurIPS 2026
Compression Breaks Discipline: Quantization Increases Missed Tool Calls in Small Language Models, but Not Mid-Size Ones
Coauthored audit of 600 controlled tool-calling generations from Qwen3-1.7B and Qwen3-4B at FP16 and NF4. Quantization raised missed required tool calls from 6.0% to 17.3% in the 1.7B model, while the 4B comparison showed no change. Error profiles exposed a failure shift that aggregate accuracy obscured.
Non-archival workshop paper.
Poster @ BlackboxNLP 2026
Inoculate or Reflect? Two Training Interventions Under Prompting, Steering, and Patching
A reproducibility study comparing inoculation prompting and counterfactual reflection training for sycophancy in Qwen3-8B. Repeated training runs test suppression, disagreement with correct users, and whether prompting or steering restores the behavior. Both interventions have limits: reflection repair overcorrects into contrarianism, and generic prompting can re-elicit sycophancy in both.
Accepted workshop paper, Reproducibility Special Track (ACL Anthology proceedings track).
Poster @ DEMI, MICCAI 2026
Subspace-Based Auditing of Bottlenecks in Retinal Fundus Foundation Embeddings
An audit of frozen DINOv2 fundus embeddings that separates disease decodability from image-quality and preprocessing signals. Patient-level splits, construction-matched nulls, and subspace removal test which signals remain recoverable. ODIR-5K provides an external check. The method is an embedding diagnostic, with limited causal claims.
Camera-ready paper available through MICCAI Open Access.
Writing
Wellbeing in Translation
Digital Minds Research Sprint report on how language changes AI-wellbeing self-report measurements across three open models. Includes a released dataset and follow-up audits. Participant research, rather than an Apart Research publication.
Companion codeInoculate or Reflect? Two training interventions under prompting, steering, and patching
Earlier write-up comparing inoculation prompting and counterfactual reflection training through behavioral and mechanistic tests. The subsequent multi-seed study was accepted to BlackboxNLP 2026; its accepted-paper abstract contains the updated results.
Companion codeAgentic Instincts: Coding Agents and their love for Pattern Matching
Case study on why AI coding agents need eval-first engineering and how coding agents should code agents.
How much of your "concept direction" is just wording?
Public writeup on wording confounds in contrastive activation extraction across 6 traits and 2 open-weight models. Companion to the Deconfounding Contrastive Interpretability project.
Companion codeYour model can't tell when it's been hacked (unless you teach it exactly how)
Public writeup on mechanism-specific limits of introspective tamper detection in Qwen3-8B. Companion to the Neuron Suppression Awareness project.
Companion codeJournal
Findings while building
More of such stuff/takes actively shared on my Twitter.
Refactoring.Guru design patterns
the best design patterns explanations, the visual elements nailed it and that cheatsheet is on point.
Open linkHenrik Karlsson on Unfolding
feedback loops over rigid plans. keep evolving :)
Open linkThe Odin Project
this got me out of tutorial hell and into shipping early on in 2021.
Open linkDistributed systems playlist
it was a tossup between this and that DSA playlist, but this one means a whole lot more to me.
Open linkCodeAesthetic
3b1b for software engineers.
Open linkLearn Git Branching
neat explanations to actually understand branching, rebasing, and merge flow.
Open linkContact
Get in touch
Reach me directly
For collaboration, technical discussion, or product ideas, email is the best channel.
Email: aaliyan1230@gmail.com