Mills regards custom as the great repressor of individuality and genius. Maybe Mills wanted a society where people could be openly homosexual and smoke opium all day if they preferred, without social approbation. I think it is more likely that the problems of an ant-like hive society of conformity that he regarded as developing in England in his time was the concern.
Farther on in chapter three Mills expresses some of the problems of mass social conformity and the way it represses individuality through pricks and goads, pressure and nodes. In some ways England is still evolving to become other-than-what-it-is 250 years after ‘On Liberty’ was composed (more or less). Mills saw England evolving the extirpation of individuality. China in that era was considered to be an example of a society with ubiquitous conformity. Being a royal subject of class superiors in England allowed for much greater yet endangered individuality than in China.. Maybe England will be renamed Little China after evolution works on English society long enough.
Mills believed free speech is useful for the advance of society and knowledge social and personal. Some free speech may be reasonably limited when it directly harms others. For instance one cannot shout ‘fire’ in a crowded theatre when there is no fire because of the stampede and loss of life that might occur. Mills writes about the relation of words to actions. Primarily words that incite select actions are wrong and may be subject to legal constrainsts are the issue. Because he supports idividuality as a summum bonum Mills takes a narrow view of limiting free speech.