He's equipped with a sub-machine gun, so backing up when opening up a suspected Crimson Shorty hideout is recommended to keep your distance. Otherwise fight him as you would fight any Crimson Lance unit. Use corrosive weapons and aim for the head.
Crimson Shorty will not drop a power core when killed.
Despite most loot midgets having high-pitched voices, the Crimson Shorty sound like normal Lance Infantry (except when he first jumps out of the chest), and wear similar armor.
On playthrough 2.5, the Crimson Shorty will have a chance of dropping level 69 SMG, shields, and grenade mods