Изтриване на MySQL с Group By

Поставете го в подзаявка:

delete from table 
where columnA in (
  select columnA
  from (
      select columnA
      from YourTable
      group by columnA
      having count(*) > 1
      ) t  
)