Conservative Muslim Forum, founder Lord Sheikh told he received “vile” Islamophobic messages after calling for Mr Johnson to be sacked.
The ex-foreign secretary had said people in burkas looked like “letter boxes” or “bank robbers”.
Elsewhere, comedian Rowan Atkinson said it had been a “pretty good” joke.
In a letter, Atkinson expressed support for Mr Johnson, saying “no apology is required” for ridiculing religion.
The party has received dozens of complaints about Mr Johnson’s comments, which he made in a Daily Telegraph column where he was also arguing against a ban on full-face veils.
Mr Johnson is now facing a possible investigation into breaches of the Conservative Party code of conduct but some Tories have criticised the party’s response to his remarks.
Former leader Iain Duncan Smith questioned the party’s handling of the row, saying: “My general view is I don’t think that an internal party system should be there to shut down MPs when they speak.
He told Today programme: “I have to say you don’t always have to agree with what people say, but we do defend the right of people saying it. Lots of people of different religions have had abusive things said in jokes by comedians but we don’t immediately shut those people down because we take offence.”
Lord Sheikh said earlier this week that Mr Johnson should be kicked out of the party for his remarks.
And speaking to Newsnight on Thursday, he added that in his view Mr Johnson had “let the genie out of the bottle” by inflaming tensions with his comments.
Out of 75 emails he had since received about the issue, 15 were complimentary while the rest included “offensive language” and “horrible things, obscene things, about the Prophet Muhammad”, Lord Sheikh told.
The complaints to the Conservative Party against Mr Johnson are being looked at by an internal party team, which will decide whether any should be looked at by an independent panel. Ultimately complaints can end up with the party’s board, which has powers including suspension or expulsion.
A source close to Mr Johnson offered no comment on the move while a Tory spokesman said: “The code of conduct process is strictly confidential.”
But Blackadder and Mr Bean creator Atkinson told the Times: “I do think that Boris Johnson’s joke about wearers of the burka resembling letterboxes is a pretty good one.
“All jokes about religion cause offence, so it’s pointless apologising for them.”
He has previously campaigned against legislation that made it an offence to incite religious hatred on the basis it could stifle freedom of speech.
And ex-Tory Chief Whip Andrew Mitchell said Mr Johnson should not have to apologise.
He said Mr Johnson had “expressed himself in quite colourful language, but he hasn’t committed any offence and we need to be very careful about our handling of this because we believe in free speech in this country”.
The Conservative Party has been accused of not doing enough to tackle anti-Muslim prejudice in its ranks, despite an initiative to boost tolerance and diversity.
The Conservative Muslim Forum and the Muslim Council of Britain had already raised concerns before Mr Johnson’s article.
A hundred Muslim women who wear the niqab or burka have signed a letter to Mr Lewis, calling on him to withdraw the Conservative whip from Mr Johnson and launch an independent inquiry into Islamophobia in the party.
“We are not forced to make these clothing choices, nor are we oppressed,” the women wrote in their letter, which has been issued to the media by the Muslim Council of Britain.
And more than 30 protesters gathered in Mr Johnson’s constituency of Uxbridge, calling for his resignation.>