Only MOV, MPEG4, MP4, AVI, WMV, MPEGPS, FLV, 3GPP, WebM formats are allowed for YouTube video uploading at this stage. If you see the "invalid file format" error when you upload video to YouTube, that means your video format is not acceptable currently. But some videos won't be uploaded successfully even though it is one of the supported format above. It is probably caused by the codec or other encoding settings. And YouTube states that the best format for YouTube to use is MP4.