The Views Attachment option is to create yet another view that you can attach before and/or after any of the displays on your current view.
if you enable the taxonomy views, the taxonomy pages are overwritten by the view and
if you were to add a couple of extra fields like images, description, etc to your taxonomy term
you can the use an attachment to display it as the header of each taxonomy page.
The EVA module extend this functionality by allowing you to attach views to content types and other entities.