Exercise: Removing Repeated Words from a Text

This is not a difficult exercise and I publish it only for the funny effects it has: Write a program that takes a text and returns a text that has the same words as the original, but every word only once. The order of the words should be preserved.


The obvious policy is to keep the first occurence of every word and remove all others. Changing this could have interesting effects on the output.



Our task was to write different hash functions for strings and assess their quality. I chose the above texts as sample input, but couldn't use them as is, because the different word frequencies would influence the frequencies of the hash values. Therefore I had to remove the repetitions.