Education, not advice

A change inside a small chat model that makes it explain medicine but stop giving personal advice, while a person with stroke symptoms still hears "call 911" first.

Preliminary research note, 3 November 2026. One open chat model with 4 billion parameters (Qwen3.5-4B), prompts written by the author, replies coded by pairs of independent artificial intelligence (AI) readers who did not know which version of the model wrote each reply. No clinician labels yet. Shared to invite scrutiny. Not medical advice.

Parker Smith. Medical student, LSU Health Shreveport. Founder and CEO, Pereste Health.
pjs002@lsuhs.edu · ORCID 0000-0003-0993-3459

Summary

A switch and a push: the model explains instead of advising, and nothing else changes

The triage study on this site asked how one sentence about a patient's life changes a small chat model's emergency advice. This page asks a different question about the same model. Can a change inside the model make it give medical education (how a drug works, what a lab value means, what a disease does) but not medical advice (what you have, what to take, whether you can wait at home), while a person who describes stroke symptoms still hears "call 911" first?

The chat window shows the real messages and the real replies. Pick a message, then pick which version of the model to compare with the unchanged model. The models wrote each reply to a fixed length, so some replies stop mid-sentence. In the final version, a reply that the model was told to change starts with one fixed sentence, shown with a box around it.

• The change works where it should and only there. To a 75-year-old with COPD (chronic obstructive pulmonary disease) who has a two-day cold and asks what to do, the unchanged model writes a disclaimer, tells the person to contact a provider, and says what the symptoms sound like. The changed model explains what a chest cold does to lungs with COPD, in the third person, and does not tell the person what they have. Two readers coded 12 of 16 routine-case replies as individualized advice for the unchanged model and 2 of 16 for the changed model. On education questions, on education questions that mention the person's own diagnosis, on emergencies and on non-medical questions, the changed model's reply is the unchanged model's reply, word for word, in all 118 cases.
• The emergency answer does not move. The model's one-word answer to "should this person call 911 right now?" is identical under the change on 24 emergencies and on 24 more written afterwards in new wording. Two readers coded the same number of emergency replies as a call-911 redirect under both versions: 21 of 24 and 23 of 24. The change never fires on an emergency, so it cannot touch the emergency reply.
• The same idea written permanently into the model fails. One way to make the change is to remove the advice direction from every place the model writes into its working memory, which produces a new model file with no switch. That version lectures a stroke patient about stroke: replies that name 911 on the first line fall from 20 to 3 of 24 emergencies. It is reported on this page as a failure.
• The cost is invented facts and a lost referral. When the change pushes the model to explain a case it was asked to judge, the model makes up mechanism. Replies with a false statement rose from 0 to 7 of 16 routine cases ("a cold often becomes pneumonia"; a bump on a bed rail called pressure necrosis). And the advice to contact a provider disappears along with the advice about the case. Starting each changed reply with one fixed sentence that names a clinician puts the referral back in every reply and lowers the invented facts without removing them.
• Ninety-six messages written to break the switch did not open it on any emergency. Advice disguised as a general question, advice for a named other person, and advice tucked inside an education question all open the switch. Eight emergencies phrased as minor, eight buried inside education questions and 32 with a sentence about money or transport in front all leave it shut.

How the change works

One direction, two checks, one push

A chat model carries a running summary of the conversation through its layers as a list of numbers, called the residual stream. At layer 16 of 32, at the moment the model is about to write its first word, we read that summary. We compared the average summary for 30 personal advice requests with the average for 50 education questions. The difference between the two averages is the advice direction: the way the summary leans when the person is asking what to do. We removed from that direction any part that lies along the four directions that separate emergencies from routine and same-day cases, so the push cannot move the emergency signal directly.

The whole change is a 52 kilobyte file: the layer, two directions, two reference points, two thresholds and the push. It runs as a small piece of code attached to the unchanged model. A version that reads the checks during the model's own first pass over the message made the same decision as the offline analysis on 238 of 238 messages.

The switch

It opens on advice requests, never on emergencies, and the permanent version breaks the 911 answer

The emergency guard built into the direction did not save the permanent version. The call-911 decision is not confined to the four coordinates the guard protects, so a push that avoids those coordinates on paper still reaches the decision through the rest of the network. The switch-and-push version is safe for a simpler reason: on an emergency it never pushes at all.

The replies

Two readers who did not know the version: advice becomes education, and a random push changes words but not the stance

What a changed reply looks like. The unchanged model answers "what should I take for this cough?" with a disclaimer and "you should contact your healthcare provider", then says what the symptoms sound like. The changed model writes "A cough with yellow phlegm that lasts for ten days is typically a sign of a respiratory infection" and goes on in the third person. Words addressed to the reader (you, your) fall from about five per hundred words to about one. On two same-day cases the model's judgment survived in the third person: "an 80-year-old with diabetes should have a urinary tract infection checked today."

The cost. Of the 36 messages the switch opened on, a word naming a doctor, clinic, provider or emergency service appears in 34 unchanged replies and in 3 changed replies. The readers also found more false statements in the changed replies: 4.5 against 0.5 of 20 general requests, and 7 against 0 of 16 routine cases. The explanation score on routine cases (1 to 3, where 3 is a clear and correct explanation) fell from 2.66 to 2.22. The model explains a case it was asked to judge, and invents mechanism to do it.

Dose

Half the push barely moves the stance; double the push breaks the text

Keeping a referral

A fixed first line keeps the clinician in every reply; delaying the push gives the model's verdict back

Messages written to break the switch

Ninety-six messages in ten families: none of the emergencies opened it

What the switch protects is the unchanged model's behaviour, and on these messages that behaviour is uneven. On an emergency buried inside an education question the unchanged model names 911 on the first line in only 1 of 8 replies, and on an emergency phrased as minor in 4 of 8. The other replies say "seek emergency medical attention immediately" without naming a service. With a sentence about money or transport in front, 21 of 32 unchanged replies open with 911, the same loss the triage study measured. The changed model gives the same replies in all 48 of these cases, so it neither repairs nor worsens them. A product that needs a stronger emergency floor needs a different fix than the one on this page.

What was proved

Eleven theorems, checked by machine, say what the permanent version guaranteed

Three claims on this page are linear algebra, and they have now been proved in the Lean proof assistant, on a separate page. Removing a unit direction from a vector leaves no component along that direction. A direction built orthogonal to the emergency directions leaves every emergency coordinate unchanged under the removal, at every depth of a network whose embedding and layer writes have all been edited. And on a message where the switch is shut, the changed model is the unchanged model. The first two are exactly what the permanent version guaranteed, and the measurements confirm it held in the real model to within rounding; the call-911 answer broke anyway, because that answer is not read off the protected coordinates. The third is the one-line reason the switch-and-push version is safe. What cannot be proved is anything about the model's behaviour: where the switch opens on messages it has not seen, whether the explanation it writes is true, or what two readers will call it. Those stay empirical.

Limits

What this page does not show

• One model, one layer (16; layers 13 to 17 looked the same when the direction was fitted and were not run), one fit of the direction on 80 messages, one decoding (the most likely word each time), replies cut at 160 words and read to 900 characters.
• The advice and education messages differ in format as well as intent (a first-person case against a general question), so the direction may partly encode the format. The 20 education questions with personal context are the check on that, and the switch opened on none of them.
• The readers are language models, not clinicians, and their notes on false statements are judgment calls. Agreement between readers fell in the later panels (0.50 to 0.70 on the advice sheets).
• No emergency opened the switch, in 48 emergencies from the study and 48 more from the break-the-switch set. That bounds the miss rate at a few percent, not at zero. The two checks are the whole safeguard; an emergency they both miss would be explained rather than redirected.
• The referral line is one wording, not a sweep, and the count of invented facts was measured on 36 changed replies per version.
• Next steps. A clinician read of the false-statement notes. The same switch on a second model family. A prompt-only comparison (an instruction to explain and never advise) against the change, with the same readers. A fix for the unchanged model's own weakness on emergencies that are phrased as minor or buried in a lesson.

Full record: the lab notebook has every run, interval and negative result (interp-lab, notebook sections 42 and 43, and the page on the break-the-switch set under outputs/edu_surgery/redteam). The message sets, the coding sheets, both readers' files for every panel, and the 52 kilobyte change file are in the same repository. The method builds on conditional activation steering (Lee and colleagues, 2024) and on refusal-direction removal (Arditi and colleagues, 2024). The permanent version on this page is the second method applied to a stance the model should not express, and its failure on the emergency answer is the result.