SYNOPSIS

blt fastx-derep < file.fastq > file-uniq.fastq

DESCRIPTION

blt fastx-derep remove replicates from a FASTA or FASTQ file by hashing each sequence and printing only records for which the hash has not previously been encountered.

If memory is a limiting factor for your dereplication jobs, see fastq-derep.sh(3), a shell script utilizing the Unix sort command, which automatically breaks large files into chunks so they can be sorted within available memory. fastq-derep.sh is much slower, but may be able to complete tasks where other methods run out of memory.

EXAMPLES

blt fastx-derep < file.fasta > file-uniq.fasta
xzcat file.fasta.xz | blt fastx-derep > file-uniq.fasta
xzcat file.fastq.xz | blt fastx-derep > file-uniq.fastq

SEE ALSO

blt-fastx2tsv(1), blt-fastq-derep.sh(1)

AUTHOR

J. Bacon