Условно в MYSQL клауза where

Можете да направите това с AND и OR стига да използвате достатъчно скоби.

Също така предполагам, че activities.OrderBy може да бъде нула. Ако това не е така, можете да премахнете нулевата проверка:

SELECT activities.*, 
       activitytypes.orderby 
FROM   activities 
       LEFT OUTER JOIN activitytypes 
                    ON activities.typeid = activitytypes.typeid 
WHERE  activities.userid = 86 
       AND activities.typeid NOT IN ( 5, 10, 11, 12, 19 ) 
       AND ( ( activities.orderby = 1 
               AND activities.starttime >= '2013-08-26 04:00:00' 
               AND activities.endtime <= '2013-08-27 04:00:00' ) 
              OR ( ( activities.orderby IS NULL 
                      OR activities.orderby != 1 ) 
                   AND activities.activitydate = '2013-08-26' ) ) 
ORDER  BY activitytypes.orderby, 
          activities.starttime 

Като алтернатива, ако все още искате да използвате CASE , просто трябва да затворите своя CASE израз с помощта на END , като това:

SELECT          activities.*, 
                activitytypes.orderby 
FROM            activities 
LEFT OUTER JOIN activitytypes 
ON              activities.typeid = activitytypes.typeid 
WHERE           activities.userid = 86 
AND             activities.typeid NOT IN ( 5, 10, 11, 12, 19 ) 
AND             ( 
                  CASE 
                  WHEN activities.orderby = 1 THEN
                    activities.starttime >= '2013-08-26 04:00:00' AND activities.endtime <= '2013-08-27 04:00:00'
                  ELSE
                    activities.activitydate = '2013-08-26' 
                  END
                ) 
ORDER BY        activitytypes.orderby, 
                activities.starttime