Don't feel too bad, he was probably expecting it and it most definitely happened to him a thousand times before and will continue to happen. It's not a big shock when you know it's going to happen, which he might have.
I speak as a bit of a thrill-seeking runner/cyclist/skater when I say this because from cycling almost everyday in the last week, my legs have become bruised and battered from the number of times I've either fallen or crashed into a wall whilst cycling too fast or making turns I shouldn't have made. When you partake in activities like cycling, skateboarding, skating, etc. you know falling and hurting is inevitable so it's nothing to you. E.g. A few days ago I cycled on a busy road for the first time in four years. I fell off my bike and on the floor twice, crashed into a wall once or twice, my phone fell out of my bag and the screen cracked, and last but definitely not least, the handlebar part on my bike loosened and changed position (which, by the way, is INCREDIBLY dangerous if it isn't fixed. Trust me, I know.). I expected all of this to happen. I was expecting the handlebar thing to happen so I prepared for it by keeping a hex key in my bag.
For me, personally, I prefer it if passers by just pass by because I know I'm going to fall and it's so much easier to move on than dwell on it by them asking me if I'm okay and stuff. I mean, I'm a clumsy individual anyway so after falling I'm used to just getting up and continue doing what I was doing.
After saying all of that, I must say that I obviously don't mean to speak for the guy but if he's skating on the street then he should be experienced enough to understand that falling is inevitable so when it happens you just pick yourself up and move on -- a little scratch and bruise is nothing. If, however, he broke a bone or something then obviously that would be a totally different story.
You're not a bad person for leaving him. I'd probably have left him too but that's because from experience I know that falling is nothing. This one time I fell so hard my knee was bleeding pretty bad. The only place I could find to sit down was a brick wall on a main road near where I live. Hundreds (in cars, on buses, bikes and walking) passed me and saw that I was bleeding but only one woman came up to me and offered a tissue because she didn't have a plaster. Although I declined I thought it was really nice of her and I appreciated it a lot but felt it was a little unnecessary because it was just a flesh wound and I was used to it.
tl;dr don't feel bad, he most probably expected to fall and is most probably used to it.