Как да получите средно с orderBy Desc в Laravel 5

Можете да използвате модифициран withCount() :

$books = Book::withCount(['ratings as average_rating' => function($query) {
    $query->select(DB::raw('coalesce(avg(rating),0)'));
}])->orderByDesc('average_rating')->get();