The funnyman will be one of several comedians who will introduce acts during Monday's Diamond Jubilee Concert.
"Comedians, including myself, Miranda Hart and Rob Brydon have foolishly agreed to link big names like Paul McCartney and Elton John by doing comedy," he told BBC Radio 2.
"I'm genuinely terrified. Why would you do that in front of the Queen? And it's worldwide.
"I go through the list of reasons in my head and there's not one redeeming feature."
He added: "It's not usual to do gigs in front of the royal family.
"I've done it – I've done the Royal Variety Show – but there's always that voice in the back of your head saying, 'Don't swear, don't swear', and that makes you want to swear."
Robbie Williams, Kylie Minogue, Stevie Wonder, The Voice coaches Sir Tom Jones and Jessie J, Ed Sheeran, Sir Elton John, Sir Paul McCartney, Dame Shirley Bassey and Annie Lennox are among the artists who will perform at The Queen's Diamond Jubilee concert.
The show takes place at the Queen Victoria memorial in front of Buckingham Palace, and will be broadcast live on BBC One and BBC Radio 2 in the UK.
Highlights will be made available on June 5 on ABC for viewers in the United States.
Lee Mack's Not Going Out co-star Tim Vine recently suggested that he has quit the sitcom.