As with any sports clothing a Downhillers garb is designed for function not fashion, unless your a footballer.
The main design considerations are protection and freedom of movement.
World Cup class Downhillers can sometimes be seen cutting about in fetching "skin suits", however most self respecting riders will wear something baggy over some form of protective clothing when racing.
Lycra will not do, nor goggles with cross-country helmets, a peak should be attached to a full face at all times. Think moto-cross and you wont go far wrong.