IBM System x3100 M4 Type 2582 Problem Determination And Service Manual page 252

10. Squeeze the spring clips on the sides of the front USB connector assembly
232
IBM System x3100 M4 Type 2582: Problem Determination and Service Guide
8. Stand the server back up in its vertical position.
9. Press down and hold the release tab on the top of the front USB housing;
then, tilt the top of the housing away from the chassis and lift the housing out
of the opening in the chassis.
and pull the assembly out of the back of the housing.
USB connector cable
Front USB housing