.. SPDX-License-Identifier: GPL-2.0+ =================== Codman code manager =================== The codman tool analyses U-Boot builds to determine which source files and lines of code are actually compiled and used. U-Boot is a massive project with thousands of files and nearly endless configuration possibilities. A single board configuration might only compile a small fraction of the total source tree. Codman can help answer questions like: * "I just enabled ``CONFIG_CMD_NET``, how much code did that actually add?" * "How much code would I remove by disabling ``CONFIG_CMDLINE``? Simply searching for ``CONFIG_`` macros or header inclusions is tricky because the build logic takes many forms: Makefile rules, #ifdefs, IS_ENABLED(), CONFIG_IS_ENABLED() and static inlines. The end result is board-specific in any case. Codman cuts through this complexity by analysing the actual build artifacts generated by the compiler: #. Builds the specified board #. Parses the ``.cmd`` files to find which source file were compiled. #. Analyses the source code (with unifdef) or the object files (dwarf tables) to figure out which files and lines were compiled. Usage ===== Basic usage, from within the U-Boot source tree:: ./tools/codman/codman.py -b [flags] [command-flags] Codman operations does out-of-tree builds, meaning that the object files end up in a separate directory for each board. Use ``--build-base`` to set that. The default is ``/tmp/cod`` meaning that a sandbox build would end up in ``/tmp/cod/sandbox``, for example. Relationship to LSPs ==================== LSPs can allow you to see unused code in your IDE, which is very handy for interactive use. Codman is more about getting a broader picture, although it does allow individual files to be listed. Codman does include a ``--lsp`` option but this doesn't work particularly well. Commands ======== The basic functionality is accessed via these commands: * ``stats`` - Show statistics (default if no command given) * ``dirs`` - Show directory breakdown * ``unused`` - List unused files * ``used`` - List used files * ``summary`` - Show per-file summary * ``detail ...`` - Show line-by-line analysis of one or more files * ``copy-used `` - Copy used source files to a directory This will build the board and show statistics about source file usage. Adjusting Configuration (-a) ============================ Sometimes you want to explore "what if" scenarios without manually editing ``defconfig`` files or running menuconfig. The ``-a`` (or ``--adjust``) option allows you to modify the Kconfig configuration on the fly before the analysis build runs. This is particularly useful for **impact analysis**: seeing exactly how much code a specific feature adds to the build. Syntax ------ The `CONFIG_` prefix is optional. * ``-a CONFIG_OPTION``: Enable a boolean option (sets to 'y'). * ``-a ~CONFIG_OPTION``: Disable an option. * ``-a OPTION=val``: Set an option (``CONFIG_OPTION``) to a specific value. * ``-a CONFIG_A,CONFIG_B``: Set multiple options (comma-separated). Examples -------- **Check the impact of USB:** Enable the USB subsystem on the sandbox board and see how the code stats change:: codman -b sandbox -a CMD_USB stats **Disable Networking:** See what code remains active when networking is explicitly disabled:: codman -b sandbox -a ~NET,NO_NET stats **Multiple Adjustments:** Enable USB and USB storage together:: codman -b sandbox -a CONFIG_CMD_USB -a CONFIG_USB_STORAGE stats Common Options ============== Building: * ``-b, --board `` - Board to build and analyse (default: sandbox, uses buildman) * ``-B, --build-dir `` - Use existing build directory instead of building * ``--build-base `` - Base directory for builds (default: /tmp/cod) * ``-n, --no-build`` - Skip building, use existing build directory * ``-a, --adjust `` - Adjust CONFIG options (see section above) Line-level analysis: * ``-w, --dwarf`` - Use DWARF debug info (most accurate, requires rebuild) * ``-i, --include-headers`` - Include header files in unifdef analysis Filtering: * ``-f, --filter `` - Filter files by wildcard pattern (e.g., ``*acpi*``) Output control: * ``-v, --verbose`` - Show verbose output * ``-D, --debug`` - Enable debug mode * ``--top `` - (for ``stats`` command) Show top N files with most inactive code (default: 20) The ``dirs command`` has a few extra options: * ``-s, --subdirs`` - Show a breakdown by subdirectory * ``-f, --show-files`` - Show individual files within directories (with ``-s``) * ``-e, --show-empty`` - Show directories/files with 0 lines used * ``-k, --kloc`` - Show line counts in kilolines (kLOC) instead of raw lines * ``--html `` - Generate an HTML report with collapsible drill-down * ``--csv `` - Generate a CSV report for spreadsheet analysis * ``-F, --files-only`` - Only output file rows in CSV (exclude directories) * ``-u, --show-unmatched`` - List files without a category match * ``-E, --show-empty-features`` - List features with no files defined Other: * ``-j, --jobs `` - Number of parallel jobs for line analysis How to use commands =================== The following commands show the different ways to use codman. Commands are specified as positional arguments after the global options. Basic Statistics (``stats``) ----------------------------- Show overall statistics for sandbox build:: $ codman -b qemu-x86 stats ====================================================================== FILE-LEVEL STATISTICS ====================================================================== Total source files: 15158 Used source files: 1172 (7.7%) Unused source files: 14105 (93.1%) Total lines of code: 3724057 Used lines of code: 243226 (6.5%) Unused lines of code: 3480831 (93.5%) ====================================================================== ====================================================================== LINE-LEVEL STATISTICS (within compiled files) ====================================================================== Files analysed: 595 Total lines in used files:261490 Active lines: 243226 (93.0%) Inactive lines: 18264 (7.0%) ====================================================================== TOP 20 FILES WITH MOST INACTIVE CODE: ---------------------------------------------------------------------- 2621 inactive lines (56.6%) - drivers/mtd/spi/spi-nor-core.c 669 inactive lines (46.7%) - cmd/mem.c 594 inactive lines (45.8%) - cmd/nvedit.c 579 inactive lines (89.5%) - drivers/mtd/spi/spi-nor-ids.c 488 inactive lines (27.4%) - net/net.c ... Directory Breakdown (``dirs``) ------------------------------ See which top-level directories contribute code:: codman dirs Output shows breakdown by directory:: ======================================================================================= Directory Files Used %Used %Code Lines Used --------------------------------------------------------------------------------------- arch 3283 132 4 3 711047 19411 board 1373 1 0 0 443368 35 boot 81 45 56 67 35432 23748 cmd 243 76 31 29 65008 18714 common 94 37 39 47 54752 25463 ... For detailed subdirectory breakdown:: codman dirs --subdirs With ``--show-files`` (``-f``), also shows individual files within each directory. Files are sorted alphabetically and those with zero active lines are hidden by default (use ``-e`` to show them):: codman dirs --subdirs --show-files Use ``--kloc`` (``-k``) for a more compact display with kilolines:: codman dirs -sfk You can also specify a file filter:: codman -b qemu-x86 -f "*acpi*" dirs -sf ======================================================================================= BREAKDOWN BY TOP-LEVEL DIRECTORY ======================================================================================= Directory Files Used %Used %Code Lines Used --------------------------------------------------------------------------------------- arch/x86/include/asm 5 2 40 36 614 221 arch/x86/lib 5 1 20 6 1166 65 acpi.c 100 65 65 cmd 1 1 100 100 216 215 acpi.c 100 216 215 drivers/qfw 1 1 100 93 332 309 qfw_acpi.c 93 332 309 include/acpi 5 4 80 91 3319 3032 include/dm 1 1 100 100 377 377 include/power 1 1 100 100 199 199 lib/acpi 13 3 23 14 3935 544 acpi.c 100 304 304 acpi_extra.c 98 181 177 acpi_writer.c 48 131 63 lib/efi_loader 1 1 100 100 75 75 efi_acpi.c 100 75 75 --------------------------------------------------------------------------------------- TOTAL 86 15 17 7 17547 1208 ======================================================================================= Detail View (``detail``) ------------------------ See exactly which lines are active/inactive in a specific file:: $ codman -b qemu-x86 detail common/main.c ====================================================================== DETAIL FOR: common/main.c ====================================================================== Total lines: 115 Active lines: 93 (80.9%) Inactive lines: 22 (19.1%) 1 | // SPDX-License-Identifier: GPL-2.0+ 2 | /* 3 | * (C) Copyright 2000 4 | * Wolfgang Denk, DENX Software Engineering, wd@denx.de. 5 | */ ... 23 | 24 | static void run_preboot_environment_command(void) 25 | { 26 | char *p; 27 | 28 | p = env_get("preboot"); 29 | if (p != NULL) { 30 | int prev = 0; 31 | - 32 | if (IS_ENABLED(CONFIG_AUTOBOOT_KEYED)) - 33 | prev = disable_ctrlc(1); /* disable Ctrl-C checking */ 34 | 35 | run_command_list(p, -1, 0); 36 | - 37 | if (IS_ENABLED(CONFIG_AUTOBOOT_KEYED)) - 38 | disable_ctrlc(prev); /* restore Ctrl-C checking */ 39 | } 40 | } 41 | Lines with a ``-`` marker are not included in the build. HTML Reports (``dirs --html``) ------------------------------ Generate an interactive HTML report with collapsible directory structure:: codman -b qemu-x86 dirs -sf --html report.html The HTML report includes: * Color-coded metrics (green for high usage, red for low) * Collapsible hierarchical directory structure for drill-down navigation * Build information in the header (board name, analysis method) * File-level details within each directory This is useful for sharing reports or exploring large codebases interactively in a web browser. CSV Reports (``dirs --csv``) ---------------------------- Generate a CSV report for spreadsheet analysis or further processing:: codman -b qemu-x86 dirs -sf --csv report.csv The CSV includes columns for Type, Path, Category, Feature, file counts, and line statistics:: Type,Path,Category,Feature,Files,Used,%Used,%Code,Lines,Used dir,arch/x86/cpu,,,20,15,75,85,3816,3227 file,arch/x86/cpu/call32.S,load-boot,boot-x86-bare,,,,100,61,61 file,arch/x86/cpu/cpu.c,load-boot,boot-x86-bare,,,,88,399,353 ... Use ``-F`` (``--files-only``) for a simplified output with just file rows (no directory summaries):: codman -b qemu-x86 dirs -sf --csv report.csv -F This produces cleaner output with columns: Path, Category, Feature, %Code, Lines, Used:: Path,Category,Feature,%Code,Lines,Used arch/x86/cpu/call32.S,load-boot,boot-x86-bare,100,61,61 arch/x86/cpu/cpu.c,load-boot,boot-x86-bare,88,399,353 arch/x86/cpu/cpu_x86.c,load-boot,boot-x86-bare,100,99,99 ... CSV reports include category information from ``category.cfg``. Other output formats (terminal, HTML) do not yet use categories. Categories and Features ----------------------- Codman can categorise source files into functional areas using the ``tools/codman/category.cfg`` configuration file. This TOML file defines: **Categories**: High-level groupings like "load-boot", "storage", "drivers" **Features**: Specific functional areas within categories, with file patterns that define which source files belong to each feature. The configuration uses three types of file patterns: * Exact paths: ``"boot/bootm.c"`` * Glob patterns: ``"drivers/video/*.c"`` * Directory prefixes: ``"lib/acpi/"`` (matches all files under the directory) Example category.cfg structure:: [categories.load-boot] description = "Loading & Boot" [features.boot-linux-direct] category = "load-boot" description = "Direct Linux boot" files = [ "boot/bootm.c", "boot/bootm_os.c", "boot/image-board.c", ] When generating HTML reports, codman matches each source file to its feature and category, making it easy to analyse code by functional area. Use ``-u`` (``--show-unmatched``) to list files that don't match any feature:: codman -b qemu-x86 dirs -sf -u Use ``-E`` (``--show-empty-features``) to list features with no files defined:: codman -b qemu-x86 dirs -sf -E **Ignoring External Code** The ``[ignore]`` section in category.cfg can exclude external/vendored code from reports:: [ignore] files = [ "lib/lwip/lwip/", # External lwIP library ] Unused Files (``unused``) ------------------------- Find all source files that weren't compiled:: $ codman -b qemu-x86 unused |head -12 Unused source files (14105): arch/arc/cpu/arcv1/ivt.S arch/arc/cpu/arcv2/ivt.S arch/arc/include/asm/arc-bcr.h arch/arc/include/asm/arcregs.h arch/arc/include/asm/bitops.h ... Used Files (``used``) --------------------- List all source files that were included in a build:: $ codman -b qemu-x86 used |head -12 Used source files (1172): arch/x86/cpu/call32.S arch/x86/cpu/cpu.c arch/x86/cpu/cpu_x86.c arch/x86/cpu/i386/call64.S arch/x86/cpu/i386/cpu.c ... Per-File Summary (``summary``) ------------------------------ Shows detailed per-file statistics (requires ``-w`` or ``-l``):: $ codman -b qemu-x86 summary ========================================================================================== PER-FILE SUMMARY ========================================================================================== File Total Active Inactive %Active ------------------------------------------------------------------------------------------ arch/x86/cpu/call32.S 61 61 0 100.0% arch/x86/cpu/cpu.c 399 353 46 88.5% arch/x86/cpu/cpu_x86.c 99 99 0 100.0% arch/x86/cpu/i386/call64.S 92 92 0 100.0% arch/x86/cpu/i386/cpu.c 649 630 19 97.1% arch/x86/cpu/i386/interrupt.c 630 622 8 98.7% arch/x86/cpu/i386/setjmp.S 65 65 0 100.0% arch/x86/cpu/intel_common/cpu.c 325 325 0 100.0% ... Copy Used Files (``copy-used``) ------------------------------- Extract only the source files used in a build:: codman copy-used /tmp/sandbox-sources This creates a directory tree with only the compiled files, useful for creating minimal source distributions. Analysis Methods ================ The script supports several analysis methods with different trade-offs. Firstly, files are detected by looking for .cmd files in the build. This requires a build to be present. Given the complexity of the Makefile rules, it seems like a reasonable trade-off. These directories are excluded: * tools/ * test/ * scripts/ * doc/ unifdef ------- For discovering used/unused code, the unifdef mechanism produces reasonable results. This simulates the C preprocessor using the ``unifdef`` tool to determine which lines are active based on CONFIG_* settings. **Note:** This requires a patched version of unifdef that supports U-Boot's ``IS_ENABLED()`` and ``CONFIG_IS_ENABLED()`` macros, which are commonly used throughout the codebase. It also supports faster operation, reducing run time by about 100x on the U-Boot code base. The tools: 1. Reads .config to extract all CONFIG_* symbol definitions 2. Generates a unifdef configuration file with -D/-U directives 3. Runs ``unifdef -k -E`` on each source file to process conditionals, with ``-E`` enabling the IS_ENABLED() support 4. Compares original vs. processed output using line-number information 5. Lines removed by unifdef are marked as inactive This method Uses multiprocessing for parallel analysis of source files, so it runs faster if you have plenty of CPU cores (e.g. 3s on a 22-thread Intel Ultra 7). The preprocessor-level view is quite helpful. It is also possible to see .h files using the ``-i`` flag Since unifdef does fairly simplistic parsing it can be fooled and show wrong results. DWARF (``-w/--dwarf``) ---------------------- The DWARF analyser uses debug information embedded in compiled object files to determine exactly which source lines generated machine code. This is arguably more accurate than unifdef, but it won't count comments, declarations and various other features that don't actually generate code. The DWARF analyser: 1. Rebuilds with ``CC_OPTIMIZE_FOR_DEBUG`` to prevent aggressive inlining 2. For each .o file, runs ``readelf --debug-dump=decodedline`` to get line info 3. Parses the DWARF line number table to map source lines to code addresses 4. Aggregates results across all object files 5. Any source line that doesn't appear in the line table is marked inactive As with unifdef, this uses multiprocessing for parallel analysis of object files. It achieves similar performance. Multi-board scanning and queries ================================ The commands above analyse a single board. Codman can also scan many boards at once and store the results in a database, so you can ask cross-board questions such as "which boards compile this file?" or "which boards still build this function?". Scanning (``scan``) ------------------- The ``scan`` command builds and analyses a set of boards, writing the results to a SQLite database (``codman.db``) in the source-tree root. Boards are selected using buildman board specifiers, so an architecture, an SoC or a wildcard all work:: codman scan sandbox codman scan arm codman scan 'qemu*' With no specifier, all boards are scanned. Each board is built with buildman and analysed with the same methods as the single-board commands. Scanning is done in parallel, with one worker per board:: codman scan -W 8 arm # use 8 workers Per-board make jobs are scaled down to match the worker count, so the machine is not oversubscribed. Use ``-1``/``--sequential`` to process boards one at a time in the main process, which is easier to debug since there is no thread or multiprocessing interaction:: codman scan -1 sandbox Other scan options: * ``--resume`` - Continue a previous scan, skipping boards already in the database. Without this, the database is recreated from scratch * ``--max-boards N`` - Stop after N boards, useful for testing * ``--exclude `` - Exclude boards matching a specifier (repeatable) * ``--clean-after`` - Delete each board's build directory once analysed, to save disk space Press Ctrl-C at any time to stop a scan cleanly; the boards scanned so far are kept in the database and ``--resume`` continues from there. Querying (``query``) -------------------- Once a scan has populated the database, the ``query`` command answers cross-board questions: * ``query file `` - Which boards compile a file (wildcards allowed) * ``query line `` - Which boards have a particular line active * ``query board [pattern]`` - Which files a board compiles * ``query unique `` - Code that only this board compiles * ``query function [file]`` - Which boards build a function * ``query info`` - Database statistics The examples below come from a scan of the sandbox boards (``codman scan sandbox``, which selects all nine sandbox variants). Use ``query info`` to see what is in the database:: $ codman query info ================================================== DATABASE INFO ================================================== Boards (OK): 9 Boards (failed): 0 Files: 1594 Board-file rows: 9606 Active ranges: 31287 Database size: 2.4 MB Architecture breakdown: sandbox 9 ``query file`` shows which boards compile a file, and how much of it each one uses. Here only three of the nine boards build ``cmd/cat.c``:: $ codman query file cmd/cat.c 3 boards compile cmd/cat.c: Board Arch Active Total % ------------------------------------------------------------------------ sandbox sandbox 71 135 52.6% sandbox64 sandbox 77 135 57.0% sandbox64_lwip sandbox 77 135 57.0% The path may contain wildcards. Combine that with ``--format count`` to get a single number - for example the total number of active lines compiled from ``cmd/`` across all the scanned boards:: $ codman query --format count file 'cmd/*' 986 ``query board`` lists the files a board compiles, optionally filtered by a pattern:: $ codman query board sandbox 'drivers/video/*' sandbox: 21 files, 7116 active lines of 7236 total (98.3%) File Active Total % ---------------------------------------------------------------------- drivers/video/backlight-uclass.c 35 35 100.0% drivers/video/bmp.c 105 141 74.5% drivers/video/console_normal.c 156 160 97.5% drivers/video/console_truetype.c 1302 1319 98.7% ... ``query unique`` finds code that only one board compiles. The lwIP variant, for instance, is the only one that pulls in the ``cmd/lwip`` commands:: $ codman query unique sandbox64_lwip sandbox64_lwip: 48988 unique lines in 92 ranges: File Lines ----------------------------------------------------------------- cmd/lwip/dhcp.c 1-9 ( 9) cmd/lwip/ping.c 1-183 (183) cmd/lwip/sntp.c 1-133 (133) ... ``query function`` reports which boards build a particular function, along with where it is defined:: $ codman query function do_bootm 7 boards build do_bootm() (cmd/bootm.c:135-174): Board Arch ---------------------------------------------------- sandbox sandbox sandbox64 sandbox sandbox_flattree sandbox ... ``query line`` answers the same question for a single source line:: $ codman query line common/main.c:42 9 boards have common/main.c:42 active All queries accept ``--arch `` to restrict the results to one architecture (useful once the database holds boards from several architectures) and ``--format table|csv|count`` to choose the output style. Line-level data is stored as run-length-encoded active ranges, which keeps the database compact even across thousands of boards. See Also ======== * :doc:`../build/buildman` - Tool for building multiple boards * :doc:`qconfig` * :doc:`checkpatch` - Code-style checking tool