So much of pacing is subjective; I know there are people who want to see action INSTANTLY and others who are happy with a build-up. If it feels less forced for you to write it one way, I'd say go with that. You mention initial minor conflict and world-building, so it doesn't sound like nothing happens at first; if that's the case I would probably prefer your original plan.