Time does not affect anything... Bosses will still have the same difficulty regardless of how long it takes to reach them.