You can use the following command to remove duplicate lines from a text file using awk:
awk '!seen[$0]++' file.txt > file_without_duplicates.txt
Alternatively, you can use the following command to achieve the same result using perl:
perl -ne 'print unless $seen{$_}++' file.txt > file_without_duplicates.txt
These commands will read the contents of file.txt line by line and store each line in a hash table. If a line has already been seen, it will not be printed to the output file file_without_duplicates.txt.