Programming in the large

1. Why JavaScript was weak at this

Frank DeRemer and Hans Kron coined the phrase in a 1975 paper, Programming-in-the-large versus programming-in-the-small, to separate writing a module from structuring a system of many modules, possibly written by different people. Traditionally, JavaScript has been weak in programming in the large, for four reasons: the lack of modules, safety issues, loose typing, and the environment it runs in, meaning browsers and the Web.

The environment is the odd one compared with languages you know. A compiled program links code its authors chose, and the linker complains about duplicate names. A web page collects scripts over the network, including analytics, advertising and chat widgets its authors did not write, and runs them all in the same space with no linker to complain. Your script has to work on a page you do not fully control, in a browser you did not pick, after a download that may fail.

Defensive Coding 101

If a script is going to play nicely on the Web, it has to be very defensive. Don't bash, and watch out for being bashed. To bash here means to overwrite something another script depends on, usually without knowing it. Encapsulating your code and assuming the worst is a good idea. These are the potential concerns, and where these pages deal with each one.

• Variable and function name conflicts. Sections 2, 4 and 5 of this page.
• Load order and network concerns. Getting script into a page and the timelines on the head page cover order. What a failure does to a module graph is covered below.
• Catastrophic errors thrown without handling. Errors and async.
• Event rebinding. Section 3 of this page.
• Browser quirks. How much JavaScript? covers feature detection.

2. One namespace for every classic script

Suppose your file defines function alertTest() { } and a variable called num. Another included script can define a function or a variable with the same name. The big challenge is that JavaScript shares its namespace with other included scripts. The possibilities are name collisions, event collisions, and misbehaving or even malicious code.

Variable collision

Code in browser-based JavaScript shares the same namespace. If you define a variable, say num, and later on a script goes and does the same, their num overwrites yours. The reverse can also happen, and with async scripts you do not get to say which one comes first. Nothing warns you. The first script still looks correct when you read it, and it now uses the other script's value.

The playground plays both scripts in one. It assigns a function to report with var, because a pair of function report() declarations in a single script would both hoist and the last would win before any line had run. In two separate files, as in the demo in the next section, each declaration takes effect when its own file runs.

That last change produces an error and none of the log lines. A top-level let or const is also global to every classic script, though it does not become a property of window, as page 03 showed. A second declaration of the name does not overwrite it. In a single script it fails to parse. Across two files, the browser throws a SyntaxError as the second file starts and runs none of that file. Switching to let makes the collision visible and costs you a whole script. A plain assignment with no declaration, num = 7, writes to whichever global already has that name, and creates one if none does.

Names you did not choose

You can also collide with the browser. The global object already holds hundreds of names, and a classic script can bash them or be changed by them.

• function alert() { } at the top level of a classic script replaces window.alert for every script on the page.
• var name = 5; gives you a string. window.name already exists and converts what it receives, so typeof name is 'string'.
• An element with id="num" makes window.num refer to that element, unless a script has declared the name. A script that reads an undeclared num gets a DOM element instead of a ReferenceError.

Misbehaving code

Every script on a page has the page's full authority. A script from a third party can read the DOM, replace fetch, or watch what a user types into a form, whether by accident or on purpose. Nothing on this page protects against that. Naming conventions and modules stop accidents between cooperating scripts. To limit a script you do not trust, you need the browser's security model, which the security model module covers, and fewer such scripts in the first place.

3. Event collision

Depending on how events are added, it is also possible to overwrite an existing event handler. A handler property such as window.onload or button.onclick holds one function, so assigning a second replaces the first.

addEventListener keeps a list. Each call adds to it, and the listeners run in the order they were added, so two scripts can both listen to one event without knowing about each other. Events, step 1 sets out the three ways to bind a handler. For this page the difference that counts is one slot against a list.

The playground creates a button and clicks it from script, so you can see both behaviors in the console. The first click runs only the chat's handler. The second click runs that handler and then both listeners, because the property takes its turn in the list at the position where it was first set. Clearing the property leaves the listeners alone.

That last suggestion shows one more rule. Adding the same function object twice for the same event and phase registers it once. Two anonymous functions with identical text are two different objects, so both register, and neither can be removed later with removeEventListener, which needs the same reference. There are other limits too. A listener added after an event has fired does not run, so a script that arrives after load and listens for it waits forever.

Both collisions on one page

This demo loads a shop script and a chat script, written as if by different people, who both chose the names num and report and both want to run on load. Each panel loads them differently.

In the first panel, with var, function and window.onload, the log shows the shop running with num at 5 and the chat running with num at 100. Then only the chat's load handler runs. The shop's handler is gone. The shop's Report button prints chat.js report: num is 100, because the shop's call to report() now finds the chat's function, which reads the chat's number.

In the second panel, both scripts declare let count at the top level. The shop runs. The chat fails with Identifier 'count' has already been declared, and not even its first line runs, though that line comes before the declaration. In the third panel the same code is written as modules using addEventListener. Each has its own num and report, each button reports its own number, and both load listeners run. addEventListener fixed the load handler and modules fixed the names, so they solve two separate problems, and section 5 explains the second.

4. Hiding from each other, before modules

Everything in this section is history. You should not write it in new code, and you will read a lot of it, in older codebases and in what bundlers produce. It also shows what modules replaced. var appears here because this code is from the time of var.

Stemming and wrapper objects

The first defense is to avoid global variables that may be bashed. If a global is necessary, make its name unlikely to collide, in one of two ways. Stemming puts a prefix on every name. A wrapper object puts all your names inside one object, so that only a single name reaches the global scope.

The wrapper is the better of the two. It uses one global name however much it holds, and it groups related code. It relies on nobody else choosing JSREF, and it hides nothing, since anyone can read or change JSREF.num. You saw the same pattern in page 03's discussion of names, as a stand-in for namespaces.

Immediately invoked function expressions

Hiding a name takes a scope, and for most of JavaScript's life only a function could make one. An immediately invoked function expression, or IIFE, defines a function and calls it on the spot. It is often used for a function that does something and then "explodes": it runs, nothing refers to it any more, and everything declared inside is out of reach of other scripts.

The outer parentheses are required. A line that starts with the word function is read as a declaration, a declaration needs a name, and it cannot be called in place. The parentheses make the parser read an expression.

The parentheses have their own trap, the one from automatic semicolon insertion. If the statement above has no semicolon, a line starting with ( continues it. This became common when files were concatenated for delivery, so older code often starts each file with a defensive ;(function () {.

IIFEs were common in ES5 and earlier versions of JavaScript. From ES6 on, the recommendation is to use modules instead. Since 2015 a bare block also hides let and const, which removes one reason for them.

The downside of an IIFE is that the call stack gets polluted with anonymous function calls all over, which makes it hard to trace. An unnamed function appears in a stack trace with no name to tell you whose it was. Giving it a name fixes that, as in (function setUpShop() { })();, and the name stays inside the function's own scope. Errors and async covers reading those traces.

The revealing module pattern

An IIFE on its own can do something and vanish, but it cannot offer anything to the rest of the page. To do more, it can use a return value or an object passed in. An object passed to a function is shared with the caller, as Functions and closures explains, so whatever the function adds to it stays visible afterward. The functions added there still reach the IIFE's local variables through a closure. Some names are revealed and the rest stay private.

The last line of the wrapper is the idiom to recognize. In a page, globalThis is window, and code of this age spelled it window.UCSD = window.UCSD || {}. It uses the object if an earlier file already created it and makes a new one if not, so any number of files can add to one namespace in any order. The one global name limits global pollution. sayHi is a public method reading a private variable, nameOfClass is a public property, and prof has no outside access. The name nameOfClass also steers around class and className, which a reader would take for the HTML attribute and its DOM property.

The other common form returns what it reveals and assigns the result to a global.

You still meet the wrapper in generated code. esbuild, for one, defaults to the iife format when it bundles for the browser, so that a bundle's variables stay out of the global scope.

5. ES modules

Much of this can go away with modern techniques such as ES6 modules. Module solutions had been around for a while, written in JavaScript or provided by a host, as with CommonJS and require in Node. ES6, the 2015 edition, made modules native to the language.

Native support then landed in browsers one at a time, starting around 2017 with Safari 10.1 and Chrome 61, and MDN dates support across browsers from May 2018, when Firefox 60 arrived. The caniuse table has current figures. You no longer need a bundler to make modules work in a browser. Reasons remain to use one, which section 6 covers. Node supports both systems: it reads .mjs files and packages marked "type": "module" as ES modules, .cjs files as CommonJS, and inspects the source when there is no marker.

What type="module" changes

ES6 defined modules, and the HTML Standard added the module type of <script>, allowing a file to encapsulate its namespace and not worry about polluting the global one. With modules you can get rid of the wrapper object.

That attribute changes more than scope. A module differs from a classic script in these ways.

• Its own top-level scope. var, let, function and class at the top level stay in the file. None of them lands on window, so var name = 5; gives you a number in a module.
• Strict mode, always. hello = 5; with no declaration throws a ReferenceError rather than creating a global, and this at the top level is undefined.
• Deferred by default. It runs after parsing, in order, inline or external. The head page's timeline shows this, and async makes it run as soon as it and its imports are ready.
• Evaluated once. However many files import a module, and even if two <script> tags name it, it runs one time and everyone shares the result. A classic script runs again each time it is included.
• Fetched with CORS, from a server. Opening the HTML file from disk fails, because a file:// URL has no usable origin. Run a local server. The server must also send a JavaScript MIME type such as text/javascript, or the browser refuses the file.

export and import

In a module, everything is private unless it is exported. Make sure your module exports properly. This file has a helper that stays private and one exported function.

In modern browsers, use <script type="module">, then in that block import things. Only a module can use the import statement. In a classic script it is a SyntaxError.

Those lines show two of the forms. The third is the named import, and you will write it most often.

A file may have many named exports and one default. Import declarations are allowed only at the top level of a module. They are hoisted, so the imported files run before the first line of the importer. A tool can therefore work out the whole dependency graph by reading the files, without running them, and that is what allows a bundler to drop exports nothing uses.

Imports are live and read-only

An import is a view of the exporter's variable, not a copy of its value. When the exporting file changes the variable, importers see the new value. Importers cannot assign to it. MDN calls these read-only live bindings. From the importer's side an import behaves like a const, and you can still change the properties of an imported object. These pages leave imported names in lowercase, because the exporting file can still change them.

A module graph you can run

The demo is one inline module importing four files, with a fifth loaded on request. Its window loads an ordinary page on this site, so you can open DevTools on it and see each file arrive in the Network panel.

Write a message to see utils.shout and fancy.write working together. Press increment() and two lines update: the page's view of count and the one in status.js. They agree because both files import the same evaluated counter.js. Press count = 99 and the page reports a TypeError, the importer being refused. The private panel lists what stayed in the files: the namespace object shows only write, fancy.makeFancy is undefined, a top-level variable in the module is not on window, and this at the top level is undefined. The last button calls import(), the function form, which returns a promise and can sit anywhere. Use it to load code only when it is needed. Errors and async covers promises. A module may also use await at its top level, which holds back anything that imports it.

Paths, and why bare names fail

The string after from is a URL or a path. In a browser it must start with /, ./ or ../, or be a full URL. Anything else is a bare specifier, and the module fails with a TypeError.

The browser asks for exactly the URL you wrote, and it has no folder of packages to search and no list of extensions to try. Tutorials are full of import x from 'some-package' because Node looks in node_modules and bundlers do the same at build time. If you do want bare names in a browser, an import map, available across browsers since March 2023, tells the page what URL each one means.

What a failure does to a module graph

Classic scripts fail one at a time. A missing or crashing file takes itself out, and later files still run, and then fail in their own way if they needed it. A module graph fails together. If an imported file cannot be fetched, or throws while it is being evaluated, the modules that import it never run. A typo in one path takes down everything above that file in the graph. You can rule out one whole family of collisions with modules, and you take on this behavior in exchange. Keep that in mind for anything optional, such as analytics, which is a good case for import() with a catch.

What modules do not fix

Modules give each file a private scope. Everything else on the page is still shared.

• window.onload = fn inside a module still replaces the handler another module set. Use addEventListener anyway.
• A module can still write window.num = 5, and it can read any global that a classic script created.
• The DOM is shared, so two modules can fight over one id or element. So are custom element names, and built-in prototypes, as Prototypes and classes warns.
• Malicious code is unaffected. A module has the same access to the page as any other script.

6. Code for yourself, prep for delivery

Developers commonly reference multiple JS files separately. With classic scripts the value of this practice is questionable, given that each file adds a network request and that the files share the same namespace. This course has a mantra for it: code for yourself, prep for delivery.

The file structure you want while working is many small files with comments, long names and tests beside them. The file structure a browser wants may be very different. Page 03 made the same point about minification. A build step is what converts the first into the second.

What has changed since the mantra

Modules answered the namespace half, since separate files no longer share top-level names. The network half has also moved. HTTP/2 carries many requests over one connection, so a request no longer costs what it did. It still costs something, and modules add a new form of the cost, since the browser discovers each level of imports only after fetching the level above it. A chain several files deep takes that many trips to find. <link rel="modulepreload">, available across browsers since September 2023, lets you name files up front, and MDN notes that listing each dependency yourself is the only way to be sure all browsers preload them.

The V8 team's 2018 article on modules recommended native modules directly for local development and for small applications with fewer than 100 modules and a shallow tree, and bundling for production otherwise. It also encouraged HTTP/2. Those numbers are from 2018, so measure your own page.

The build tool the mantra had in mind has changed as well. Gulp was the usual example of a task runner, and later Webpack of a bundler. Vite, esbuild and Rollup are the names you will now see alongside Webpack. This course does not teach any of them, and their shared job is what to understand. They follow the imports, combine files, remove unused exports and minify the result. When a build step takes over, the file on the server stops resembling the file you wrote, so source maps connect the two for debugging.

Whether you need a build step

It Depends™, as it does for loading. A site with a handful of modules can ship them as written, with nothing to install and nothing between source and browser. A large application with hundreds of files and packages from npm cannot reasonably do that. Start with native modules and no tools, and add a build step when the measured performance or the dependencies give you a reason.

Reduce rather than add

To be blunt, when we encounter a problem, our main approach is often to add more JavaScript, to solve problems we may have created with JavaScript or because JavaScript didn't quite work as we expected. Wrapper objects, loaders and bundlers each began as an answer of that kind. The aim in this course is to see if we can reduce complexity rather than add it, since less complex things are more efficient and often more robust. How much JavaScript?, the next page, makes that argument in full.

7. @ts-check: type checking without TypeScript

Loose typing was the third item on the list in section 1. Nothing at the point of use tells you what a variable holds, as page 03 showed with the line that joined a number to a string. TypeScript addresses this with a new syntax and a compile step. You can get many of its benefits without either.

Adding a simple // @ts-check comment to the top of any JS file in VS Code gives you many of the benefits of TypeScript without having to compile from a .ts file or learn TS syntax. Other editors that use the TypeScript language service can do the same.

The two benefits shown are the ones to start with. It flags a change to the data type of a variable after its declaration, and it flags assigning to a variable that has no declaration at all, such as hello = 5 where hello has not been declared. Hovering over the name shows the type it worked out, here let numError: number. It checks more than these two, including misspelled properties and querySelector results that may be null.

It warns and does not prevent

The checker reads your file in the editor and marks problems with a squiggle. The browser never sees the comment as anything but a comment, so the code still runs. Flagged code ships unless you run the checker somewhere that can stop it.

In a classic script that hello = 5 also creates a global, so the warning catches the same accident that section 2 worried about.

The other comments

With no comment and no project setting, a .js file is not type checked. // @ts-nocheck marks a file as unchecked explicitly. It changes something only when checking is on for a whole project, where it opts one file out. To turn checking on for every JavaScript file, use "js/ts.implicitProjectConfig.checkJs": true in VS Code's settings, or a jsconfig.json for the project.

// @ts-ignore disables the check just for the next line. In this file the type change on line 5 is no longer flagged, while hello on line 6 still is.

// @ts-expect-error does the same, with one addition: if the next line has no error, the comment itself is reported. That makes it the better choice, since it tells you when the reason for it has gone away.

What it infers, and what you can add

The checker infers a variable's type from its initial value, so a variable given a number is a number from then on. Without a useful initial value it has nothing to go on. Neither variable below gets a warning, however often its type changes. A JSDoc comment supplies a type where inference has none, and it gives a function's parameters a contract that callers are checked against. It is still only a comment, so it adds nothing to the running code.

It also understands the subject of this page. Two checked files that have no import or export are treated as scripts sharing one global scope, so a let num in each is reported as Cannot redeclare block-scoped variable 'num', the collision from the second demo panel found before anything runs. Add an import or export to make them modules, and that report goes away. An assignment to an imported name is reported as well, as Cannot assign to 'num' because it is an import.

To make the check count for a team, run it outside the editor. This command, tried here with TypeScript 7.0.2, checks every listed file and writes nothing. A failing exit code from it can stop a build.

The comment does not make JavaScript a typed language. Values can still arrive from a form or a server with a type nothing checked.

8. Summary