Select all the topics you want, left join all the relevant comments, group them by topic ID and make one of your rows a count of that ID.
Code:
SELECT
t.id.
t.author,
... (additional info you want from the topic table here)
c.forumtopicid,
COUNT(t.id) as reply_count
FROM
forum_topics t
LEFT JOIN
forum_comments c ON c.forumtopicid=t.id
GROUP BY
t.id
WHERE
t.category=1
If this query does not make sense see what it looks like without the GROUP BY statement and it might help.