No, in most cases it's spending time to save time. When an IDispose
class has a finalizer, its Dispose methods typically runs the exact
same code that the finalizer runs, plus it calls GC.SuppressFinalize.
This is saving all the CPU costs involved in running the finalizer,
plus it allows the memory to be reused sooner.

When an IDispose class does not have a finalizer, it usually has
private reference(s) to handle class(es) that do have a finalizer, so
the call to Dispose allows it to close its handle(s), thus (again)
saving all the CPU costs involved in running the handle finalizer(s),
plus allowing the memory to be reused sooner.

The only time calling Dispose doesn't save time is with the minority
of IDispose implementations that aren't avoiding a finalizer, that are
doing something like restoring the GUI cursor or noting how long it's
been since a stopwatch class has been created.