You may commit a crime in various ways: by picking someone’s pocket (your chances will depend on your stealth skill but if other citizens are near your chances will be worse), by breaking into certain homes (some houses can be broken into day or night if you have a skeleton key – some windows that are not blinking at night can be broken into with your weapon), by harming a peaceful civilian or creature and by stealing. If you attempt to break into a house during the day, a citizen may catch you and call the guards. At night, you can try to by-pass the guards.
Unless you bypass the guards, you will be arrested – you will then have to pay a fine or work off your time. In jail, you may try to lockpick your way out if your lockpick skill is high enough and you have a lockpick. You also can spend your time in prison (try to pace up and down to pass the time) and then get out, but then some of your attributes and/or skills will decrease.
Lockpicks and skeleton keys can be purchased in shady shops or from certain individuals throughout the island.
You can also find people who will clear your crime rating for money.