Deleting a custom avatar removes its uploaded source image and built assets, and it will no longer appear in your avatar list.
You will not be able to start new sessions with that avatar, but past session and usage records remain. Stock avatars cannot be deleted.