Introduction to WebKit

What is WebKit?

WebKit is an open-source Web browser engine. It’s a framework in macOS and iOS, and used by many first party and third party applications including Safari, Mail, Notes, Books, News, and App Store.

The WebKit codebase is mostly written in C++ with bits of C and assembly, primarily in JavaScriptCore, and some Objective-C to integrate with Cocoa platforms.

It primarily consists of the following components, each inside its own directory in Source:

  • bmalloc - WebKit’s malloc implementation as a bump pointer allocator. It provides an important security feature, called IsoHeap, which segregates each type of object into its own page to prevent type confusion attacks upon use-after-free.
  • WTF - Stands for Web Template Framework. WebKit’s template library. The rest of the WebKit codebase is built using this template library in addition to, and often in place of, similar class templates in the C++ standard library. It contains common container classes such as Vector, HashMap (unordered), HashSet, and smart pointer types such as Ref, RefPtr, and WeakPtr used throughout the rest of WebKit.
  • JavaScriptCore - WebKit’s JavaScript engine; often abbreviated as JSC. JSC parses JavaScript and generates byte code, which is then executed by one of the following four tiers. Many tiers are needed to balance between compilation time and execution time. Also see Phil's blog post about Speculation in JavaScriptCore.
    • Interpreter - This tier reads and executes instructions in byte code in C++.
    • Baseline JIT - The first Just In Time compiler tier serves as the profiler as well as a significant speed up from the interpreter.
    • DFG JIT - Data Flow Graph Just In Time compiler uses the data flow analysis to generate optimized machine code.
    • FTL JIT - Faster than Light Just In Time compiler which uses B3 backend. It’s the fastest tier of JSC. JavaScriptCode also implements JavaScriptCore API for macOS and iOS applications.
  • WebCore - The largest component of WebKit, this layer implements most of the Web APIs and their behaviors. Most importantly, this component implements HTML, XML, and CSS parsers and implements HTML, SVG, and MathML elements as well as CSS. It also implements CSS JIT, the only Just In Time compiler for CSS in existence. It works with a few tree data structures:
    • Document Object Model - This is the tree data structure we create from parsing HTML.
    • Render Tree - This tree represents the visual representation of each element in DOM tree computed from CSS and also stores the geometric layout information of each element.
  • WebCore/PAL and WebCore/platform - Whilst technically a part of WebCore, this is a platform abstraction layer for WebCore so that the rest of WebCore code can remain platform independent / agnostic across all the platforms WebKit can run on: macOS, iOS, Windows, Linux, etc... Historically, most of this code resided in WebCore/platform. There is an ongoing multi-year project to slowly migrate code to PAL as we remove the reverse dependencies to WebCore.
  • WebKitLegacy (a.k.a. WebKit1) - This layer interfaces WebCore with the rest of operating systems in single process and implements WebView on macOS and UIWebView on iOS.
  • WebKit (a.k.a. WebKit2) - This layer implements the multi-process architecture of WebKit, and implements WKWebView on macOS and iOS. WebKit’s multi-process architecture consists of the following processes:
    • UI process - This is the application process. e.g. Safari and Mail
    • WebContent process - This process loads & runs code loaded from websites. Each tab in Safari typically has its own WebContent process. This is important to keep each tab responsive and protect websites from one another.
    • Networking process - This process is responsible for handling network requests as well as storage management. All WebContent processes in a single session (default vs. private browsing) share a single networking session in the networking process.
  • WebInspector / WebDriver - WebKit’s developer tool & automation tool for Web developers.

Contributing to WebKit

There are many ways to get involved and contribute to the WebKit Project. Filing a new bug, fixing a bug, or adding a new feature.

There are three different kinds of contributors in the WebKit project.

  • Contributor - This category encompasses everyone. Anyone who files a bug or contributes a code change or reviews a code change is considered as a contributor
  • Committer - A committer is someone who has write access to WebKit's subversion repository.
  • Reviewer - A reviewer is someone who has the right to review and approve code changes other contributors proposed.

See Commit and Review Policy for more details on how to become a committer or a reviewer.

Staying in Touch

Before getting in touch with WebKit developers using any of the avenues below, make sure that you have checked our page on how to ask questions about WebKit.

You can find WebKit developers, testers, and other interested parties on the #WebKit Slack workspace. Join the WebKit slack, and stay in touch.

Bug tracking in WebKit

bugs.webkit.org hosted is the primary bug tracking tool we use. When making a code change, we post a code change (patch) on this website.

Filing a bug and editing bugs

To file a new WebKit bug, see reporting bugs.

To edit an existing bug, you may need editbug-bits.

Code Reviews in bugs.webkit.org

We also use bugs.webkit.org to upload & review code changes to WebKit. You can post a code change with Tools/Scripts/webkit-patch upload. Note that the webkit-patch script only looks for changes below current directory, so generally you should change the current directory to the top-level directory of a WebKit first.

When a patch is posted on bugs.webkit.org requesting a formal code review (r? flag is set), The Early Warning System (a.k.a. EWS) will automatically build and run tests against your code change. This allows contributors to find build or test failures before committing code changes to the WebKit’s primary Subversion repository.

Once a patch is approved by a reviewer (r+ flag is set), then the patch can be either committed directly into the Subversion repository by a WebKit committer, who has write access to the Subversion repository, or via the commit queue which can be requested by setting cq? flag and approved by any WebKit committer by setting cq+.

The Subvesion commit message should be created by Tools/Scripts/commit-log-editor based on the change log entries. Tools/Scripts/webkit-patch land does this automatically.

Security Bugs in bugs.webkit.org

Security bugs have their own components in bugs.webkit.org. We’re also working on a new policy to delay publishing tests for security fixes until after the fixes have been widely deployed.

***Do not post a patch or describe a security bug in a bug that is not in security component of bugs.webkit.org.***

Getting started with WebKit

Getting Code

See Getting the Code

Adding Tools to PATH

For convenience, you can add Tools/Scripts/ to your path as follows in ~/.zshrc like so:

export PATH=$PATH:/Volumes/Data/webkit/Tools/Scripts/

where /Volumes/Data/webkit is the path to a WebKit checkout.

This will allow you to run various tools you by name instead of typing the full path of the script.

Updating checkouts

There is a script to update a WebKit checkout: Tools/Scripts/update-webkit. This script will automatically merge change logs mentioned below.

Building WebKit

See Building WebKit

Fixing mysterious build or runtime errors after Xcode upgrades

If you see mysterious build failures or if you’ve switched to a new version of macOS or Xcode, delete the WebKitBuild directory. make clean may not delete all the relevant files, and building after doing that without deleting the WebKitBuild directory may result in mysterious build or dyld errors.

Building with Address Sanitizer to investigate memory corruption bugs

To build Address Sanitizer or ASan builds to analyze security bugs, run Tools/Scripts/set-webkit-configuration --asan --release. This will enable ASan build. If want to attach a debugger, you can also specify --debug instead of --release. Once you don’t need to build or run ASan anymore, you can specify --no-asan in place of --asan to disable ASan. Note that this configuration is saved by creating a file called Asan in the WebKitBuild directory, so if you are trying to do a clean Asan build by deleting the build directory you need to rerun this command.

Using Xcode

You can also use Xcode to build & debug WebKit. Open WebKit.xcworkspace at the top level directory.

In order to make Xcode use build files built by make command above, go to File > Workspace Settings... > Advanced... > Custom > Relative to Workspace and adjust the relative paths of Products and Intermediates to point to WebKitBuild directory. Screenshot of XCode Workspace Settings Screenshot of XCode Workspace Settings - Advanced Build Location Note that debugging WebCore code typically requires attaching to the relevant WebContent process, not the application process, which is mostly running code in Source/WebKit/UIProcess. Depending on what you’re debugging, you’d have to attach & debug different processes in the coalition.

You may find it useful to use the debug helpers under Tools/lldb/lldb_webkit.py. This can be added to ~/.lldbinit for automatic loading into LLDB on launch by adding the line command script import {Path to WebKit}/Tools/lldb/lldb_webkit.py. For more details, see the Wiki article on lldb formatters.

When debugging a debug build in LLDB, there are also a few functions that can be called on objects that will dump debugging info.

  • RenderObject
    • showNodeTree()
    • showLineTree()
    • showRenderTree()
  • Node
    • showTree()
    • showNodePath()
    • showTreeForThis()
    • showNodePathForThis()

Correctness Testing in WebKit

WebKit is really big on test driven development, we have many types of tests.

  • JavaScript tests - Resides in top-level JSTests directory. This is the primary method of testing JavaScriptCore. Use Tools/Scripts/run-javascriptcore-tests to run these tests.
  • Layout tests - Resides in top-level LayoutTests directory. This is the primary method of testing WebCore. If you’re making code changes to WebCore, you typically run these tests. Use Tools/Scripts/run-webkit-tests to run these. Pass -1 to run tests using WebKitLegacy (a.k.a. WebKit1). WebKitTestRunner is used to run these tests for WebKit2, and DumpRenderTree is used to these tests for WebKit1. There are a few styles of layout tests but all of them have a test file and expected result (ends with -expected.txt), and the test passes if the test file’s output matches that of the expected result.
  • API tests - Reside in Tools/TestWebKitAPI, these are GTests that test APIs exposed by JavaScriptCore, WebKitLegacy, and WebKit layers as well as unit tests for selected WTF classes. WebKit does not use XCTests. Use Tools/Scripts/run-api-tests to run these tests. Because these API tests are sequentially, it’s preferable to write layout tests when possible.
  • Bindings tests - Reside in Source/WebCore/bindings/scripts/test, these are tests for WebCore’s binding code generator. Use Tools/Scripts/run-bindings-tests to run these tests.
  • webkitpy tests - Tests for WebKit’s various Python scripts in Tools/Scripts/webkitpy. Use Tools/Scripts/test-webkitpy to run these tests.
  • webkitperl tests - Tests for WebKit’s various Perl scripts in Tools/Scripts/webkitperl. Use Tools/Scripts/test-webkitperl to run these tests.

Performance Testing in WebKit

The WebKit project has a “no performance regression” policy. We maintain the performance of the following of the benchmarks and are located under PerformanceTests. If your patch regresses one of these benchmarks even slightly (less than 1%), it will get reverted.

  • JetStream2 - Measures JavaScript and WASM performance.
  • MotionMark - Measures graphics performance.
  • Speedometer 2 - Measures WebKit’s performance for complex web apps.

The following are benchmarks maintained by Apple‘s WebKit team but not available to other open source contributors since Apple doesn’t have the right to redistribute the content. If your WebKit patch regresses one of these tests, your patch may still get reverted.

  • RAMification - Apple's internal JavaScript memory benchmark.
  • ScrollPerf - Apple's internal scrolling performance tests.
  • PLT - Apple's internal page load time tests.
  • Membuster / PLUM - Apple's internal memory tests. Membuster for macOS and PLUM for iOS and iPadOS.

Contributing code to WebKit

WebKit has a rigorous code contribution process and policy in place to maintain the quality of code.

Coding style

Code you write must follow WebKit’s coding style guideline. You can run Tools/Scripts/check-webkit-style to check whether your code follows the coding guidelines or not (it can report false positives or false negatives). If you use Tools/Scripts/webkit-patch upload to upload your patch, it automatically runs the style checker against the code you changed so there is no need to run check-webkit-style separately.

Some older parts of the codebase do not follow these guidelines. If you are modifying such code, it is generally best to clean it up to comply with the current guidelines.

Convenience Tools

Tools/Scripts/webkit-patch provides a lot of utility functions like applying the latest patch on bugs.webkit.org (apply-from-bug) and uploading a patch (upload --git-commit=<commit hash>) to a bugs.webkit.org bug. Use --all-commands to the list of all commands this tool supports.

Licensing

Much of the code we inherited from KHTML is licensed under LGPL. New code contributed to WebKit will use the two clause BSD license. When contributing new code, update the copyright date. When moving the existing code, you need to include the original copyright notice for the moved code and you should also not change the license, which may be BSD or LGPL depending on a file, without the permission of the copyright holders.

Regression Tests

Once you have made a code change, you need to run the aforementioned tests (layout tests, API tests, etc...) to make sure your code change doesn’t break existing functionality. These days, uploading a patch on bugs.webkit.org triggers the Early Warning System (a.k.a. EWS)

For any bug fix or a feature addition, there should be a new test demonstrating the behavior change caused by the code change. If no such test can be written in a reasonable manner (e.g. the fix for a hard-to-reproduce race condition), then the reason writing a tests is impractical should be explained in the accompanying change log.

Any patch which introduces new test failures or performance regressions may be reverted. It’s in your interest to wait for the Early Warning System to fully build and test your patch on all relevant platforms.

ChangeLog Files

ChangeLogs are simple text files which provide historical documentation for all changes to the WebKit project. All patches require an entry to the ChangeLog.

The first line contains the date, your full name, and your email address. Use this to write up a brief summary of the changes you’ve made. Don’t worry about the “Reviewed by NOBODY (OOPS!)” line, the person landing your patch will fill this in. There is one ChangeLog per top-level directory. If you changed code and tests you will need to edit at least two ChangeLogs. Tools/Scripts/prepare-ChangeLog script will create stub entries for ChangeLog files based on code changes you made in your Git or Subversion checkouts.

You should edit these stubs to describe your change, including the full URL to the bug (example entry, note that you can use --bug flag). (You should set EMAIL_ADDRESS and CHANGE_LOG_NAME in your environment if you will be running this script frequently.) A typical change log entry before being submitted to bugs.webkit.org looks like this:

2012-10-04  Enrica Casucci  <e•••••@apple.com>

        Font::glyphDataAndPageForCharacter doesn't account for text orientation when using systemFallback on a cold cache.
        https://bugs.webkit.org/show_bug.cgi?id=98452.

        Reviewed by NOBODY (OOPS!).

        The text orientation was considered only when there is a cache hit.
        This change moves the logic to handle text orientation to a separate
        inline function that is called also when the glyph is added to the cache.

        Test: fast/text/vertical-rl-rtl-linebreak.html

        * platform/graphics/FontFastPath.cpp:
        (WebCore::applyTextOrientationForCharacter): Added.
        (WebCore::Font::glyphDataAndPageForCharacter): Modified to use the new function in
        both cases of cold and warm cache.

The “No new tests. (OOPS!)” line appears if prepare-ChangeLog did not detect the addition of new tests. If your patch does not require test cases (or test cases are not possible), remove this line and explain why you didn’t write tests. Otherwise all changes require test cases which should be mentioned in the ChangeLog.

WebKit’s Build System

Apple’s macOS, iOS, watchOS, and tvOS ports use Xcode and the rest use CMake to build WebKit. There is an ongoing effort to make Apple's ports also use CMake.

In order to reduce the compilation time, which used to take 40+ minutes on the fully loaded 2018 15“ MacBook Pro, we bundle up multiple C++ translation units (.cpp files) and compile them as a single translation unit. We call this mechanism Unified Sources or Unified Builds.

Unified sources are generated under WebKitBuild/X/DerivedSources where X is the name of build configuration such as Debug and Release-iphonesimulator. For example, WebKitBuild/Debug/DerivedSources/WebCore/unified-sources/UnifiedSource116.cpp may look like this:

#include "dom/Document.cpp"
#include "dom/DocumentEventQueue.cpp"
#include "dom/DocumentFragment.cpp"
#include "dom/DocumentMarkerController.cpp"
#include "dom/DocumentParser.cpp"
#include "dom/DocumentSharedObjectPool.cpp"
#include "dom/DocumentStorageAccess.cpp"
#include "dom/DocumentType.cpp"

How to add a new .h or .cpp file

To add a new header file or a translation unit (e.g. .cpp, .m, or .mm), open WebKit.xcworkspace and add respective files in each directory.

Make sure to uncheck the target membership so that it’s not compiled as a part of the framework in xcodebuild. Instead, add the same file in Sources.txt file that exists in each subdirectory of Source. e.g. Source/WebCore/Sources.txt for WebCore. This will ensure the newly added file is compiled as a part of unified sources. Screenshot of adding a file to Xcode When a header file in WTF is used in WebCore, or a header file in WebCore is used in WebKit or WebKitLegacy, we need to export the file to those projects. To do that, turn on the target membership in respective framework as set the membership to “Private” as seen below. This will ensure the relevant header file is exported from WTF / WebCore to other downstream projects like WebKitLegacy.