Download the source code here: samtools-1.23.tar.bz2.(The "Source code" downloads are generated by GitHub and are incomplete as they don't bundle HTSlib and are missing some generated files.)

New work and changes:

• New reference stats in samtools stats. First line in RFS section gives the total sequence count, count of regions, average GC, min, max, average and total counts. Second line onwards gives regions, lengths, GC and unknown base count. (PR #2224, implements #2139. Requested by Filipe G. Vieira)

• New, faster Python version of seq_cache_populate to create and update REF_CACHE. (PR #2231. Thanks to Ruben Vorderman)

• Add a minimum depth (--min-depth) option to samtools coverage. (PR #2235, implements #1563. Requested by Charles Foster)

• Add an option to exclude reads (--exclude-no-read-group) that have no read group from samtools view when the -r (or -R) options are used. (PR #2271, fixes #2265. Reported by Matt Sexton)

• Add UMI support to samtools fastq and samtools import. See samtools/htslib#1960. (PR #2270, fixes #2259 amd #2262. Requested by Poshi)

• Optionally trim soft clips from reads in samtools fastq output. (PR #2233, fixes #1275. Requested by Torsten Seemann)

• If sam file is sorted by tag, samtools split will output data sequentially to avoid having simultaneous open files. (PR #2281, fixes #2276. Requested by Clint Valentine)

Documentation:

• In the command help output add a link to the global options in the samtools.1 page on the HTSlib site. (PR #2258, addresses #2236. Reported by Chris Saunders)

• Add a support section to README.md. This mentions the GitHub issue tracker and an email address for security issues. (PR #2267)

Bug fixes:

• Prevent samtools coverage from printing a coverage table on failure. (PR #2247, fixes #2242. Reported by Georges Kanaan)

• Remove deprecated line style commands from plot-bamstats. (PR #2251, fixes #2243. Reported by Suhas Srinivasan)

• Add missing sam_global_args_free() calls to address (harmless) memory leaks. (PR #2274)

• Fix samtools consensus crash when used with threads and iterators. See also samtools/htslib#1959 (PR #2269)

Non user-visible changes and build improvements:

• Ignore and testclean test/stat/*.fa.fai (PR #2241. Thanks to John Marshall)

• Remove use of C variables starting in _ from bam_consensus.c. (PR #2250, fixes #2248. Reported by Ghanji125)

• Add Replace RG check exit and add some comments to bam_addrprg.c. (PR #2254. Thanks to Martin Pollard)

Download the source code here: samtools-1.23.tar.bz2.(The "Source code" downloads are generated by GitHub and are incomplete as they don't bundle HTSlib and are missing some generated files.)

Download the source code here: samtools-1.22.1.tar.bz2.(The "Source code" downloads are generated by GitHub and are incomplete as they don't bundle HTSlib and are missing some generated files.)

Bug fixes:

• SECURITY: Fix use-after-free bug in samtools mpileup -a due to reference sequences being discarded too early. This could happen when the -a option was used, and all the alignments for one of the references started at the same position. This caused mpileup to try to load the next reference before it had finished writing data for the previous one out. (PR #2229, fixes #2227. Reported by Pouya Kheradpour)

• This release bundles htslib-1.22.1 and htscodecs v1.6.4. Fixes a possible buffer overflow on some invalid CRAM inputs; and a failure when trying to save data with very long alignment records with sequence '*' as CRAM version 3.1.
  See the HTSlib and htscodecs NEWS files for details.

Documentation:

• Clarify that -@ starts additional threads in samtools index help, and fix formatting issues in the samtools reset manual page. (PR #2225)

Build changes:

• Added settings to limit memory used by samtools sort when running tests. (PR #2226)

Download the source code here: samtools-1.22.1.tar.bz2.(The "Source code" downloads are generated by GitHub and are incomplete as they don't bundle HTSlib and are missing some generated files.)

Download the source code here: samtools-1.22.tar.bz2.(The "Source code" downloads are generated by GitHub and are incomplete as they don't bundle HTSlib and are missing some generated files.)

Note this release changes the default output CRAM version from 3.0 to 3.1. HTSlib and SAMtools have been able to read CRAM 3.1 since version 1.12, however other tools may not yet be able to cope. We know Noodles reads CRAM3.1 and htsjdk has a draft implementation, but not yet released.
HTSlib has options for modifying the output formats, which are exposed in SAMtools. When specifying an output format you can explicitly change the version via e.g. samtools view -O cram,version=3.0 ....
Further documentation on this change can be found at https://www.htslib.org/benchmarks/CRAM.html

HTSlib no longer fetches CRAM reference data from EBI's server by default. Your organisation may wish to set up local infrastructure to supply reference sequences, e.g., using the new ref-cache tool included in this HTSlib release. See the REF_CACHE and REF_PATH environment variables documented in https://www.htslib.org/doc/reference_seqs.html and the SAMtools manpage for details.

New work and changes:

• New samtools checksum command. This checksums sequence, name, quality and barcode tags in an order and orientation agnostic manner, to facilitate validation of no data loss between raw fastq (or unmapped crams) through alignment, duplication marking, sorting, and other processing operations to get to the final aligned bam/cram. (PR #2122)

• Extend samtools sort -M to distinguish between mapped and unmapped files. (PR #2110, fixes #2105. Reported by Armin Töpfer)

• Allow the samtools sort "merging from..." message to be silenced. Setting the verbosity to 0 or 1 will now silence this message. (PR #2197, resolves #2185. Requested by Alex Predeus)

• Add --save-counts option to samtools view. Adds an option to store counts of records processed, accepted and rejected by filtering to a file. (PR #2120, resolves #2038. Requested by Chang Y)

• samtools fasta and fastq can now make faidx/fqidx indexes while writing using the --write-index option. (PR #2125, resolves #2118. Requested by Filipe G. Vieira)

• Add a warning for samtools fastq on coordinate sorted data. (PR #2176, fixes #2169 and #2161. Reported by wook2014)

• samtools tview add -i to hide inserts. (PR #2123. Thanks to Benjamin Bræstrup Sayoc)

• Show optional headers with samtools bedcov -H. (PR #2140, fixes #2126. Reported by biounix)

• samtools consensus now supports proper multi-threading. Previously this was restricted to decompression only, but it should now scale better. (PR #2174, supersedes PR #2141)

• Add samtools consensus -T ref.fa functionality. This reports the reference value if a consensus value cannot be calculated. (PR #2153, fixes an additional request in #1915)

• In samtools consensus, do not use consensus N for "*" (absent) calls that are masked due to insufficient depth. (PR #2204, fixes #2167. Reported by sanschiffre)

• Improve plot-bamstats quality plots. (PR #2143 combined with PR #2116 (thanks to James Gilbert))

• Make reheader -h use /tmp and honour TMPDIR. (PR #2168, related to #2165. Reported by Zhang Yuanfeng)

• Set sort order header tag to unsorted when ordering is lost during samtools merge. (PR #2173, fixes #2159. Reported by Filipe G. Vieira)

• Protect against merging CRAM files with different headers. (PR #2220, fixes #2218. Reported by Kevin Lewis)

• samtools stats bug-fix to checksum calculation for quality values. This corrects the checksums but in turn makes the calculated value different to that reported by previous samtools versions. (PR #2193, fixes #2187)

• Clarification for samtools stats when used on files with different sort orders. (PR #2198, fixes #2177. Reported by Filipe G. Vieira)

• In samtools stats, dovetailed (completely overlapping) read pairs are now always counted as inward-oriented. Previously they could have been inwards or outwards depending on read ordering. (PR #2216, resolves #2210. Requested by Pontus Hüer)

Documentation:

• Correct the example for 1:1 samtools consensus coords. (PR #2113, fixes #2111. Reported by schorlton-bugseq)

• Documents the fastq format options used in SAMtools and HTSlib. (PR #2123, fixes #2121)

• Remove mention of threads from samtools cat man page. (PR #2162, fixes #2160. Reported by Brandon Pickett)

• Update samtools merge man page to include --template-coordinate. (PR #2164. Thanks to Nils Homer)

• Revised CRAM reference sequence documentation in the samtools man page. (PR #2178)

• Added fish shell completion and renamed completion for bash shell. These files can be copied to appropriate directories by the user. For full functionality it requires Python3.5+ and installed samtools manpages. (PR #2203. Thanks to LunarEclipse363)

• Fix URL printed by the seq_cache_populate.pl script. (PR #2222. Thanks to Charles Plessy)
  Bug fixes:

• samtools consensus previously could give different results for BAM and CRAM files with the same content. This was because MD/NM tag generation was disabled in CRAM, but the decode_md=0 option did nothing with BAM. Note with --no-adj-MQ both BAM and CRAM gave identical results. Now use --input-fmt-option decode_md=0 to get the old CRAM behaviour. Otherwise, both BAM and CRAM will be utilising MD/NM to locally modify mapping quality. (PR #2156)

• samtools consensus without -a previously still padded with leading Ns in some cases. It now consistently removes both leading and trailing Ns. Use "-a" if you want all reference bases displayed. (Part of PR #2174 above)

• Change how markdup looks for single reads. Due to changes to fixmate in 1.21 markdup no longer recognised single reads that would have normally have been part of a pair. (PR #2117, fixes #2117. Reported by Kristy Horan)

• Fix samtools merge crash on BAM files with malformed headers. (PR #2128, fixes #2127. Reported by Frostb1te)

• Fix faidx --write-index invalid free. (PR #2147, fixes 2142. Reported by Alex Leonard)

• Fix samtools fastq -i to force CRAM aux tag decoding. (PR #2155, fixes #2155. Reported by Alex Leonard)
  Non user-visible changes and build improvements:

• Improve htslib#PRnum support for Cirrus-CI and GitHub Actions. (PR #2115)

• Fix broken tests due to MSYS2 changes. Due to changes in how MSYS2 perl reported the identity of the OS it was built for, our tests were failing to adapt to the Windows style file locations. (PR #2196)

• Upgrade to _XOPEN_SOURCE=700, to match HTSlib. Also replace usleep() with nanosleep(). (PR #2221)

Download the source code here: samtools-1.22.tar.bz2.(The "Source code" downloads are generated by GitHub and are incomplete as they don't bundle HTSlib and are missing some generated files.)

Download the source code here: samtools-1.21.tar.bz2.

Notice:

• This is the last SAMtools / HTSlib release where CRAM 3.0 will be the default CRAM version. From the next we will change to CRAM 3.1 unless the version is explicitly specified, for example using samtools view -O cram,version=3.0.

New work and changes:

• samtools reset now removes a set of predefined auxtags, as these tags are no longer valid after the reset operation. This behaviour can be overridden if desired. (PR #2034, fixes #2011. Reported by Felix Lenner)

• samtools reset now also removes duplicate flags. (PR #2047. Reported by Kevin Lewis)

• Region and section/part filtering added to CRAM samtools cat. Region filtering permits samtools cat to produce new CRAMs that only cover a specified region. (PR #2035)

• Added a report of the number of alignments for each primer to samtools ampliconclip. (PR #2039, PR #2101, feature request #2033. Thanks to Brad Langhorst)

• Make ampliconclip primer counts output deterministic. (PR #2081)

• samtools fixmate no longer removes the PAIRED flag from reads that have no mate. This is done on the understanding that the PAIRED flag is a sequencing technology indicator not a feature of alignment. This is a change to previous fixmate behaviour. (PR #2056, fixes #2052. Reported by John Wiedenhoeft)

• Added bgzf compressed FASTA output to samtools faidx. (PR #2067, fixes #2055. Requested by Filipe G Vieira)

• Optimise samtools depth histogram incrementing code. (PR #2078)

• In samtools merge zero pad unique suffix IDs. (PR #2087, fixes #2086. Thanks to Chris Wright)

• samtools idxstats now accepts the -X option, making it easier to specify the location of the index file. (PR #2093, feature request #2071. Requested by Samuel Chen)

• Improved documentation for the mpileup --adjust-MQ option. (PR #2098. Requested by Georg Langebrake)

Bug fixes:

• Avoid tview buffer overflow for positions with >= 14 digits. (PR #2032. Thanks to John Marshall. Reported on bioconda/bioconda-recipes#47137 by jmunoz94)

• Added file name and error message to 'error closing output file' error in samtools sort. (PR #2050, fixes #2049. Thanks to Joshua C Randall).

• Fixed hard clip trimming issue in ampliconclip where right-hand side qualities were being removed from left-hand side trims. (PR #2053, fixes #2048. Reported by Duda5)

• Fixed a bug in samtools merge --template-coordinate where the wrong heap was being tested. (PR #2062. Thanks to Nils Homer. Reported on ng-core/fastquorum#52 by David Mas-Ponte)

• Do not look at chr * for unmapped-placed reads with samtools view --fetch-pairs. This was causing a significant slowdown when --fetch-pairs was being used. (PR #2070, fixes #2059. Reported by acorvelo)

• Fixed bug which could cause samtools view -L to give incomplete output when the BED file contained nested target locations. (PR #2107, fixes #2104. Reported by geertvandeweyer)

• Enable samtools coverage to handle alignments that do not have quality score data. This was causing memory access problems. (PR #2083, fixes #2076. Reported by Matthew Colpus)

• Fix undefined behaviour in samtools fastq with empty QUAL. (PR #2084)

• In plot-bamstats fixed read-length plot for data with limited variations in length. Lack of data was causing gnuplot problems. (PR #2085, fixes #2068. Reported by mariyeta)

• Fixed an accidental fall-through that caused samtools split -p to also enable --no-PG. (PR #2101)

• Fixed an overflow that caused samtools consensus -m simple to give incorrect output when the input coverage exceeded several million reads deep. (PR #2099, fixes #2095. Reported by Dylan Lawrence)

Non user-visible changes and build improvements:

• Work around address sanitizer going missing from the Cirrus CI ubuntu clang compiler by moving the address sanitizer build to gcc. Fix warnings from the new clang compiler. (PR #2043)

• Windows based CI has been migrated from AppVeyor to GitHub Actions. (PR #2072, PR #2108)

• Turn on more warning options in Cirrus-CI builds, ensure everything builds with -Werror, and add undefined behaviour checks to the address sanitizer test. (PR #2101, PR #2103, PR #2109)

• Tidy up Makefile dependencies and untracked test files. (PR #2106. Thanks to John Marshall)

Download the source code here: samtools-1.21.tar.bz2. (The "Source code" downloads are generated by GitHub and are incomplete as they don't bundle HTSlib and are missing some generated files.)

Download the source code here: samtools-1.20.tar.bz2.(The "Source code" downloads are generated by GitHub and are incomplete as they don't bundle HTSlib and are missing some generated files.)

New work and changes:

• Added a --max-depth option to bedcov, for more control over the depth limit used when calculating the pileup. Previously this limit was set at 64000; now it is set to over 2 billion, so effectively all bases will be counted. (PR #1970, fixes #1950. Reported by ellisjj)

• Added mpileup --output-extra RLEN to display the unclipped read length. (PR #1971, feature request #1959. Requested by Feng Tian)

• Improved checking of symbolic flag names (e.g. UNMAP) passed to samtools. (PR #1981, fixes #1977. Reported by Ilya Shlyakhter)

• The samtools consensus --min-depth option now works for the Bayesian mode as well as the simple one. (PR #1989, feature request #1982. Requested by Gautier Richard)

• It's now possible to use the samtools fastq -d tag:val option multiple times, allowing matches on more than one tag/value. It also gets a -D option which allows the values to be listed in a file. (PR #1993, feature request #1958. Requested by Tristan Lefebure)

• Added samtools fixmate -M option to sanity check base modification (ML, MM, MN) tags, and where necessary adjust modification data on hard-clipped records. (PR #1990)

• Made mpileup run faster. (PR #1995)

• samtools import now adds a @PG header to the files it makes. As with other sub-commands, this can be disabled by using --no-PG. (PR #2008. Requested by Steven Leonard)

• The samtools split -d option to split by tag value now works on tags with integer values. (PR #2005, feature request #1956. Requested by Alex Leonard)

• Adjusted samtools sort -n (by name) so that primary reads are always sorted before secondary / supplementary. (PR #2012, feature request #2010. Requested by Stijn van Dongen)

• Added samtools bedcov -H option to print column headers in the output. (PR #2025. Thanks to Dr. K. D. Murray)

Documentation:

• Added a note that BAQ is applied before filtering and overlap removal during mpileup processing. (PR #1988, fixes #1985. Reported by Joseph Galasso)

• Added 3.1 to the list of supported CRAM versions in the samtools manual page. (PR #2009. Thanks to Andrew Thrasher)

• Made assorted improvements to ampliconclip, flagstat and markdup manual pages. (PR #2014)

Bug Fixes:

• Security fix: Fixed double free that could occur if bed file indexing failed due to running out of memory. This bug first appeared in version 1.19.1. (PR #2026)

• Corrected error message printed when faidx fails to load the fai index. (PR #1987. Thanks to Nick Moore)

• Fixed bug introduced in release 1.4 that caused incorrect reference bases to be printed by samtools mpileup -a -f ref.fa in the zero-depth regions at the end of each reference. (PR #2019, fixes #2018. Reported by Joe Georgeson)

• Fixed a samtools view usage crash on MinGW when given invalid options. (PR #2030, fixes #2029. Reported by Divon Lan)

Non user-visible changes and build improvements:

• Added tests to ensure that CRAM compression is working properly. (PR #1969, part of fix for #1968. Reported by Clockris)

Download the source code here: samtools-1.19.2.tar.bz2.(The "Source code" downloads are generated by GitHub and are incomplete as they don't bundle HTSlib and are missing some generated files.)

Bug Fixes:

• Fixed a regression in 1.19.1 that broke BED filtering for inputs where the region start positions for the same reference were not sorted in ascending order. (PR #1975, fixes #1974. Reported by Anže Starič)

Download the source code here: samtools-1.19.1.tar.bz2.(The "Source code" downloads are generated by GitHub and are incomplete as they don't bundle HTSlib and are missing some generated files.)

Bug Fixes:

• Fixed a possible array bounds violation when looking up regions in a BED file (e.g. using samtools view -L). This could lead to crashes or the return of incomplete results if the BED file contained a large number of entries all referencing low positions on a chromosome. (PR #1962, fixes #1961. Reported by geertvandeweyer)

• Fixed a crash in samtools stats that occurred when trying to clean up after it was unable to open a CRAM reference file. (PR #1957, fixes crash reported in samtools/htslib#1723. Reported by Alex Leonard)
  Documentation:

• Fixed inverted logic in the samtools consensus --show-del manual page description. (PR #1955, fixes #1951. Reported by Mikhail Schelkunov)

• Added a description of the MPC section to the samtools stats manual page. (PR #1963, fixes #1954. Reported by litun-fkby)

Download the source code here: samtools-1.19.tar.bz2.(The "Source code" downloads are generated by GitHub and are incomplete as they don't bundle HTSlib and are missing some generated files.)

New work and changes:

• Samtools coverage: add a new --plot-depth option to draw depth (of coverage) rather than the percentage of bases covered. (PR #1910. Thanks to Pierre Lindenbaum)

• Samtools merge / sort: add a lexicographical name-sort option via the -N option. The "natural" alpha-numeric sort is still available via -n. (PR #1900, fixes #1500. Reported by Steve Huang)

• Samtools view: add -N ^NAME_FILE and -R ^RG_FILE options. The standard -N and -R options only output reads matching a specified list of names or read-groups. With a caret (^) prefix these may be negated to only output read not matching the specified files. (PR #1896, fixes #1895. Suggested by Feng Tian)

• Cope with Htslib's change to no longer close stdout on hts_close(). Htslib companion PR is samtools/htslib#1665. (PR #1909. Thanks to John Marshall)

• Plot-bamstats: add a new plot of the read lengths ("RL") from samtools stats output. (PR #1922, fixes #1824. Thanks to erboone, suggested by Martin Pollard)

• Samtools split: support splitting files based on the contents of auxiliary tags. Also adds a -M option to limit the number of files split can make, to avoid accidental resource over-allocation, and fixes some issues with --write-index. (PR #1222, PR #1933, fixes #1758. Thanks to Valeriu Ohan, suggested by Scott Norton)

Bug Fixes:

• Samtools stats: empty barcode tags are now treated as having no barcode. (PR #1929, fixes #1926. Reported by Jukka Matilainen)

• Samtools cat: add support for non-seekable streams. The file format detection blocked pipes from working before, but now files may be non-seekable such as stdin or a pipe. (PR #1930, fixes #1731. Reported by Julian Hess)

• Samtools mpileup -aa (absolutely all positions) now produces an output even when given an empty input file. (PR #1939. Reported by Chang Y)

• Samtools markdup: speed up optical duplicate tagging on regions with very deep data. (PR #1952)

Documentation:

• Samtools mpileup: add more usage examples to the man page. (PR #1913, fixes #1801)

• Samtools fastq: explicitly document the order that filters apply. (PR #1907)

• Samtools merge: fix example output to use an uppercase @RG PL field. (PR #1917. Thanks to John Marshall. Reported by Michael Macias)

• Add hclen SAM filter documentation. (PR #1902. See also samtools/htslib#1660)

• Samtools consensus: remove the -5 option from documentation. This option was renamed before the consensus subcommand was merged, but accidentally left in the man page. (PR #1924)

Download the source code here: samtools-1.18.tar.bz2.(The "Source code" downloads are generated by GitHub and are incomplete as they don't bundle HTSlib and are missing some generated files.)

New work and changes:

• Add minimiser sort option to collate by an indexed fasta. Expand the minimiser sort to arrange the minimiser values in the same order as they occur in the reference genome. This is acts as an extremely crude and simplistic read aligner that can be used to boost read compression. (PR #1818)

• Add a --duplicate-count option to markdup. Adds the number of duplicates (including itself) to the original read in a dc tag. (PR #1816. Thanks to wulj2)

• Make calmd handle unaligned data or empty files without throwing an error. This is to make pipelines work more smoothly. A warning will still be issued. (PR #1841, fixes #1839. Reported by Filipe G. Vieira)

• Consistent, more comprehensive flag filtering for fasta/fastq. Added --rf/--incl[ude]-flags and long options for -F (--excl[ude]-flags and -f (--require-flags). (PR #1842. Thanks to Devang Thakkar)

• Apply fastq --input-fmt-option settings. Previously any options specified were not being applied to the input file. (PR #1855. Thanks to John Marshall)

• Add fastq -d TAG[:VAL] check. This mirrors view -d and will only output alignments that match TAG (and VAL if specified). (PR #1863, fixes #1854. Requested by Rasmus Kirkegaard)

• Extend import --order TAG to --order TAG:length. If length is specified, the tag format goes from integer to a 0-padded string format. This is a workaround for BAM and CRAM that cannot encode an order tag of over 4 billion records. (PR #1850, fixes #1847. Reported by Feng Tian)

• New -aa mode for consensus. This works like the -aa option in depth and mpileup. The single a reports all bases in contigs covered by alignments. Double aa (or -a -a) reports Ns even for the references with no alignments against them. (PR #1851, fixes #1849. Requested by Tim Fennell)

• Add long option support to samtools index. (PR #1872, fixes #1869. Reported by Jason Bacon)

• Be consistent with rounding of "average length" in samtools stats. (PR #1876, fixes #1867. Reported by Jelinek-J)

• Add option to ampliconclip that marks reads as unmapped when they do not have enough aligned bases left after clipping. Default is to unmap reads with zero aligned bases. (PR #1865, fixes #1856. Requested by ces)

Bug Fixes:

• [From HTSLib] Fix a major bug when searching against a CRAM index where one container has start and end coordinates entirely contained within the previous container. This would occasionally miss data, and sometimes return much more than required. The bug affected versions 1.11 to 1.17, although the change in 1.11 was bug-fixing multi-threaded index queries. This bug did not affect index building. There is no need to reindex your CRAM files. (PR #samtools/htslib#1574, PR #samtools/htslib#1640. Fixes #samtools/htslib#1569, #samtools/htslib#1639, #1808, #1819. Reported by xuxif, Jens Reeder and Jared Simpson)

• Fix a sort -M bug (regression) when merging sub-blocks. Data was valid but in a poor order for compression. (PR #1812)

• Fix bug in split output format. Now SAM and CRAM format can chosen as well as BAM. Also a documentation change, see below. (PR #1821)

• Add error checking to view -e filter expression code. Invalid expressions were not returning an error code. (PR #1833, fixes #1829. Reported by Steve Huang)

• Fix reheader CRAM output version. Sets the correct CRAM output version for non-3.0 CRAMs. (PR #1868, fixes #1866. Reported by John Marshall)

Documentation:

• Expand the default filtering information on the mpileup man page. (PR #1802, fixes #1801. Reported by gevro)

• Add an explanation of the default behaviour of split files on generating a file for reads with missing or unrecognised RG tags. Also a small bug fix, see above. (PR #1821, fixes #1817. Reported by Steve Huang)

• In the INSTALL instructions, switched back to openssl for Alpine. This matches the current Alpine Linux practice. (PR #1837, see htslib#1591. Reported by John Marshall)

• Fix various typos caught by lintian parsers. (PR #1877. Thanks to Étienne Mollier)

• Document consensus --qual-calibration option. (PR #1880, fixes #1879. Reported by John Marshall)

• Updated the page about samtools duplicate marking with more detail at https://www.htslib.org/algorithms/duplicate.html
  Non user-visible changes and build improvements:

• Removed a redundant line that caused a warning in gcc-13. (PR #1838)

Download the source code here: samtools-1.17.tar.bz2.(The "Source code" downloads are generated by GitHub and are incomplete as they don't bundle HTSlib and are missing some generated files.)

New work and changes:

• New samtools reset subcommand. Removes alignment information. Alignment location, CIGAR, mate mapping and flags are updated. If the alignment was in reverse direction, sequence and its quality values are reversed and complemented and the reverse flag is reset. Supplementary and secondary alignment data are discarded. (PR #1767, implements #1682. Requested by dkj)

• New samtools cram-size subcommand. It writes out metrics about a CRAM file reporting aggregate sizes per block "Content ID" fields, the data-series contained within them, and the compression methods used. (PR #1777)

• Added a --sanitize option to fixmate and view. This performs some sanity checks on the state of SAM record fields, fixing up common mistakes made by aligners. (PR #1698)

• Permit 1 thread with samtools view. All other subcommands already allow this and it does provide a modest speed increase. (PR #1755, fixes #1743. Reported by Goran Vinterhalter)

• Add CRAM_OPT_REQUIRED_FIELDS option for view -c. This is a big speed up for CRAM (maybe 5-fold), but it depends on which filtering options are being used. (PR #1776, fixes #1775. Reported by Chang Y)

• New filtering options in samtools depth. The new --excl-flags option is a synonym for -G, with --incl-flags and --require-flags added to match view logic. (PR #1718, fixes #1702. Reported by Dario Beraldi)

• Speed up calmd's slow handling of non-position-sorted data by adding caching. This uses more memory but is only activated when needed. (PR #1723, fixes #1595. Reported by lxwgcool)

• Improve samtools consensus for platforms with instrument specific profiles, considerably helping for data with very different indel error models and providing base quality recalibration tables. On PacBio HiFi, ONT and Ultima Genomics consensus qualities are also redistributed within homopolymers and the likelihood of nearby indel errors is raised. (PR #1721, PR #1733)

• Consensus --mark-ins option. This permits he consensus output to include a markup indicating the next base is an insertion. This is necessary as we need a way of outputting both consensus and also how that consensus marries up with the reference coordinates. (PR #1746)

• Make faidx/fqidx output line length default to the input line length. (PR #1738, fixes #1734. Reported by John Marshall)

• Speed up optical duplicate checking where data has a lot of duplicates compared to non-duplicates. (PR #1779, fixes #1771. Reported by Poshi)

• For collate use TMPDIR environment variable, when looking for a temporary folder. (PR #1782, based on PR #1178 and fixes #1172. Reported by Martin Pollard)

Bug Fixes:

• Fix stats breakage on long deletions when given a reference. (PR #1712, fixes #1707. Reported by John Didion)

• In ampliconclip, stop hard clipping from wrongly removing entire reads. (PR #1722, fixes #1717. Reported by Kevin Xu)

• Fix bug in ampliconstats where references mentioned in the input file headers but not in the bed file would cause it to complain that the SAM headers were inconsistent. (PR #1727, fixes #1650. Reported by jPontix)

• Fixed SEGV in samtools collate when no filename given. (PR #1724)

• Changed the default UMI barcode regex in markdup. The old regex was too restrictive. This version will at least allow the default read name UMI as given in the Illumina example documentation. (PR #1737, fixes #1730. Reported by yloemie)

• Fix samtools consensus buffer overrun with MD:Z handling. (PR #1745, fixes #1744. Reported by trilisser)

• Fix a buffer read-overflow in mpileup and tview on sequences with seq "*". (PR #1747)

• Fix view -X command line parsing that was broken in 1.15. (PR #1772, fixes #1720. Reported by Francisco Rodríguez-Algarra and Miguel Machado)

• Stop samtools view -d from reporting meaningless system errors when tag validation fails. (PR #1796)

Documentation:

• Add a description of the samtools tview display layout to the man page. Documents . vs , and upper vs lowercase. Adds a -s sample example, and documents the -w option. (PR #1765, fixes #1759. Reported by Lucas Ferreira da Silva)

• Clarify intention of samtools fasta/q in man page and soft vs hard clipping. (PR #1794, fixes #1792. Reported by Ryan Lorig-Roach)

• Minor fix to wording of mpileup --rf usage and man page. (PR #1795, fixes #1791. Reported by Luka Pavageau)

Non user-visible changes and build improvements:

• Use POSIX grep in testing as egrep and fgrep are considered obsolete. (PR #1726, thanks to David Seifert)

• Switch MacOS CI tests to an ARM-based image. (PR #1770)