Does Dispose method deletes the connection object from the memory?

No, it doesn’t delete it from memory, it closes any resources (e.g. returns the underlying database connection to the connection pool).
The garbage collector actually will delete the object from memory at some undefined point in the future.