On this page

History · Line endings

CR, LF and the CRLF wars

Two separate teletype motions, and why your diffs still argue about them.

Change one line of a file, open the diff, and sometimes every line in the file has changed. Other times one line looks the same as before, except for a red ^M at the end. Both come from a decision made for machines with paper in them: moving to a new line takes two separate motions.

ASCII gives each motion its own code. CR (carriage return, 0x0D) goes back to the start of the line. LF (line feed, 0x0A) moves down one line. Every system since has had to decide whether a line ends with both codes, or with one code doing the work of two. They didn’t all decide the same way.

System or protocol Ends a line with Notes
Teleprinter codes (ITA2) CR and LF Two codes for two motions
Multics LF, read as “new line” CR not in its set
Unix, Linux, macOS LF The terminal driver adds the CR
Older Mac applications CR OS X uses LF
CP/M CR LF
Windows CR LF Notepad’s default for new files
Telnet CR LF CR NUL for a bare carriage return
SMTP (email) CR LF Nothing else allowed
HTTP/1.1 CR LF A lone LF may be accepted
Git, for text files LF in the repository Can be CR LF in the working copy
Table 1 How different systems and protocols end a line.

Two motions

On a teleprinter, the two jobs belonged to different parts of the machine. The carriage carried the print position across the paper, and at the end of a line it had to travel all the way back to the left margin. Feeding the paper up a line was a separate mechanism. Teleprinter codes older than ASCII, such as ITA2, already had a code for each.

The return was the slow part. A carriage was a moving metal part that needed tens of milliseconds to get back to the margin, and the next character had to wait until it got there. The fix was padding: the sender followed CR with characters that printed nothing, such as NUL, to give the carriage time to arrive.

ASCII kept both codes as format effectors. CR moves the printing position “to the first printing position on the same printing line,” and LF moves it “to the next printing line.” On paper the order of the two didn’t matter: CR then LF and LF then CR left the print head in the same place.

A clause for one code

The 1968 version of the standard allowed a shortcut. “Where appropriate,” it says, LF may mean New Line (NL), moving “to the first printing position on the next printing line.” That is both motions in a single code. There was a condition: “Use of this convention requires agreement between sender and recipient of data.”

The clause made either answer legitimate. A system could send CR and LF, or send LF alone and treat it as a new line, as long as both ends agreed. Much of what followed comes from systems that never agreed.

Multics takes the shortcut

Multics was a time-sharing system, a joint project of MIT, Bell Labs and General Electric. In a 1970 paper, J. H. Saltzer of MIT and J. F. Ossanna of Bell Labs described how it handled text going to and from terminals. Their plan was to “choose a small subset of the possible control characters, give them precise meanings, and attempt to honor those meanings on every device, by interpretation if necessary.”

Their table defines 11 of the 33 control characters. LF becomes NL: “Move carriage to left edge of next line.” CR isn’t on the list. A program ended a line with one character, and the system translated it for whatever terminal was attached. The paper gives a similar example for a new page: on a Model 37 Teletype the system sent a form feed and a carriage return, while on an IBM 2741 it sent enough new lines to reach the next page.

Ossanna also belonged to the Bell Labs group that began looking for an alternative to Multics in 1969, as the Labs withdrew from the project. The others were Ken Thompson, Dennis Ritchie and Doug McIlroy. Their system, Unix, used the same rule. Its 1979 terminal driver could map a typed CR into LF and echo either one back as a carriage return and line feed, so programs saw only the new-line character.

Small computers keep both

Personal computers took the other path. The manual for CP/M, an operating system for early microcomputers, describes text files as “a sequence of ASCII characters, separated by end-of-line characters (a carriage return/line-feed sequence).” Windows still ends lines that way.

For many years, Windows Notepad read nothing else. Until 2018, Microsoft wrote, Notepad “was unable to correctly display the contents of text files created in Unix, Linux and macOS.” In 2018 Microsoft announced that Notepad would read LF and CR files too, while still using CR LF for new files.

Apple chose a third answer. Many older Mac applications ended each line with CR alone. OS X is a Unix system, and its command-line tools use LF. Apple’s scripting guide explains what happens when a Unix tool reads an old Mac file: everything shows up as one line, with each line overwriting the last, because a carriage return moves the cursor back to the left edge without moving it down. CR does exactly what it was built to do.

The wire speaks CRLF

Network protocols settled on both codes. Telnet’s 1983 specification noted that “many systems and terminals do not treat CR and LF independently” and required the sequence CR LF to be treated as a single new line. A sender that really meant a bare carriage return had to send CR NUL instead, with the NUL once again standing in as filler.

Email is the strictest. SMTP requires every line to end with CR LF, and conforming implementations “MUST NOT recognize or generate any other character or character sequence as a line terminator.” The standard explains why: a bare CR or LF “has a long history of causing problems in mail implementations.” HTTP/1.1 is more forgiving. Senders use CRLF, but a recipient “MAY recognize a single LF as a line terminator and ignore any preceding CR.”

Line endings were still causing trouble in December 2023, when Timo Longin of SEC Consult published SMTP smuggling, a way to send spoofed email that relied on mail servers disagreeing about where a message ends. A message ends with a line containing only a full stop, which should be CR LF, full stop, CR LF. Some servers also accepted versions built from bare line endings, and others passed those versions along untouched. When the sending server read the full stop as part of the message and the receiving server read it as the end, whatever followed was read as new SMTP commands, and so as a new email from whatever sender the attacker chose.

Why your diffs still argue

Git sits in the middle of all this. For files marked as text, Git normalizes line endings to LF when they are added to the repository and, depending on settings and platform, may convert them to CR LF when it writes them out again. If a file’s line endings change, whether in an editor or because two computers convert them differently, every line now ends differently, so every line has changed, and the diff shows the whole file removed and added back.

When only some lines gain a CR, the diff shows them with ^M. That’s caret notation: CR is Ctrl-M, because M is 64 code points above CR in the table. Why Ctrl works that way has its own article. By default, Git counts the stray CR as whitespace at the end of the line and highlights it in red on added lines. A setting can tell Git to treat it as part of the line ending instead, but it’s off unless someone turns it on.

Both codes still do what they were built to do. Send a lone CR to a terminal and the cursor returns to the start of the line, ready to write over it. That’s how a command-line progress bar redraws itself in place. LF still moves down a line. Unix-family systems stopped asking for both. Windows and the wire still do.

Continue reading

Sources

  1. Vint Cerf, “ASCII format for Network Interchange,” RFC 20, October 1969, which reproduces the code and control definitions of USAS X3.4-1968. https://www.rfc-editor.org/rfc/rfc20
  2. Tom Jennings, “An annotated history of some character codes,” including the entries for CR and LF. https://www.sr-ix.com/Archive/CharCodeHist/index.html
  3. J. H. Saltzer and J. F. Ossanna, “Remote Terminal Character Stream Processing in Multics,” preprint, March 1970, for the 1970 Spring Joint Computer Conference. https://www.multicians.org/M0121.pdf — Dennis M. Ritchie, “The Evolution of the Unix Time-sharing System,” 1979. https://www.nokia.com/bell-labs/about/dennis-m-ritchie/hist.html — Unix Seventh Edition manual, “tty(4).” https://man.cat-v.org/unix_7th/4/tty — and Linux man-pages, “termios(3).” https://man7.org/linux/man-pages/man3/termios.3.html
  4. Digital Research, CP/M Operating System Manual (CP/M 2.2), §1.6.5 “ED Command.” http://www.cpm.z80.de/manuals/cpm22-m.pdf — Apple, “Designing Scripts for Cross-Platform Deployment,” Shell Scripting Primer. https://developer.apple.com/library/archive/documentation/OpenSource/Conceptual/ShellScripting/PortingScriptstoMacOSX/PortingScriptstoMacOSX.html — and Michel Lopez, “Introducing extended line endings support in Notepad,” Microsoft, May 2018. https://devblogs.microsoft.com/commandline/extended-eol-in-notepad/
  5. J. Postel and J. Reynolds, “Telnet Protocol Specification,” RFC 854, May 1983. https://www.rfc-editor.org/rfc/rfc854 — J. Klensin, “Simple Mail Transfer Protocol,” RFC 5321, §2.3.8, October 2008. https://www.rfc-editor.org/rfc/rfc5321 — and R. Fielding, M. Nottingham and J. Reschke, “HTTP/1.1,” RFC 9112, §2.2, June 2022. https://www.rfc-editor.org/rfc/rfc9112
  6. Git documentation, “gitattributes” (the text and eol attributes) and “git-config” (core.autocrlf and core.whitespace). https://git-scm.com/docs/gitattributes and https://git-scm.com/docs/git-config — and Timo Longin, “SMTP Smuggling – Spoofing E-Mails Worldwide,” SEC Consult, December 2023. https://sec-consult.com/blog/detail/smtp-smuggling-spoofing-e-mails-worldwide/