Azeem Rafiq has barely touched a cricket ball since he last played professionally five years ago. That’s why the recent picture of him leaving the field is so striking. Leaner and fitter than he’s ever been, the 33-year-old strides towards the camera with gloves in one hand, bat in the other, and the gleeful smile of someone who has just scored runs.

He had had no intention of playing for the Authors XI in their Hay festival fixture against the local side. “When I got asked, I was like, oh my God, no!” says Rafiq, who had to borrow both kit and gear from his teammates. “But the minute I turned up, it brought back memories of when I first started playing – the little club ground with its football-style dressing room. You couldn’t have wiped the smile off my face.”

Rediscovering any joy in cricket has come as a surprise. In November 2021, he wept as he told a parliamentary hearing how his treatment in the game had driven him to the brink of suicide. His evidence of institutional racism at Yorkshire county cricket club, where Asian players were regularly called “Paki”, led the news headlines for days and triggered a wave of inquiries and investigations across the sport. The English game has been reckoning with the fallout ever since.

But the personal consequences for Rafiq have been just as severe. Since the moment he stepped before the digital, culture, media and sport (DCMS) committee, he has faced relentless abuse, attacks and death threats. “My life changed over that hour and 45 minutes,” he says in his soft Barnsley accent. His new memoir, It’s Not Banter, It’s Racism, recounts some of the worst moments: the human excrement left on his parents’ lawn, the chain-wielding man who stalked his house in the middle of the night.

“I felt like something really bad was going to happen,” he says today. He and his wife, Faryal, have a four-year-old son, Ayaan, and a three-year-old daughter, Mirha. “My kids, they’re babies, they don’t need to go through that stuff.” His sister Amna, who worked at Yorkshire as a development manager, left her role when she was told the club could “no longer guarantee her safety”. In late 2022 Rafiq, along with his parents, children and Amna, left their home of two decades and moved to Dubai. This three-week book tour is the longest he has been back in the UK since.

Rafiq no longer needs the three-man security team that the English cricket governing body assigned him in the aftermath of his whistleblowing, but he still avoids travelling north. During a short stopover in Barnsley last summer two separate men tried to pick a fight with him in the town centre. Speaking out has cost him friends, relationships and work opportunities, and rebuilding his life is an ongoing project.

“I still battle. I’ve struggled with my mental health, which I’ve been very open about for a very long time. And there’s days and weeks where it feels like, God, I just need a break, I need a hug.” He is making a living through public speaking engagements and consultancy work for the law firm that represented him in his legal cases. There’s some media work too, such as covering Sunday’s match between Pakistan and India at the T20 World Cup in New York.

You sense a growing confidence and urbanity in his identity as an anti-racism campaigner – the Benjamin Zephaniah quotes, the references to Channel 4 documentaries. He has often been told that “whistleblowers never have a successful life after”, something he is “pretty determined to change”. But even this book has had a difficult route to publication. A first version was co-written with George Dobell, the cricket journalist who brought his story to national attention. When Dobell and the publishers disagreed on its editorial direction, he pulled out and Rafiq had to begin again.

It hasn’t affected their friendship: the two men still speak every day. Dobell himself has been subject to threats and intimidation since he began reporting Rafiq’s case but has no regrets: “All the times we’ve talked, millions of hours, he’s never said anything that didn’t stack up,” he says. “We might have differences of opinion at times, but I’ve found him to be scrupulously honest.”

Rafiq is careful about the work he takes on – “everything I do has a responsibility to the cause” – and he knows how closely he will be scrutinised. The day after he gave evidence to MPs, antisemitic tweets he had posted as a 19-year-old were unearthed, and threatened to undermine his testimony. His response – an immediate and contrite apology, followed by a programme of self-education that included time with Jewish leaders and a journey to Auschwitz – was a model one.

Then there are the individuals who reach out to him for help with their experiences of discrimination. “If it’s in the news, you can have five or six in a week, and it doesn’t just come from sport. You get it from all sorts of places, people working in banks, or local authorities. One of the worst ones that I got was a phone call from a guy working in an anti-racism charity.” His book’s subtitle is What Cricket’s Dirty Secret Reveals About Our Society, and he saves some of his most vituperative comments for politicians such as Rishi Sunak, Suella Braverman and Priti Patel.

“Rishi was contacted several times by people in my case and he has had no interest in it,” says Rafiq. “I think it’s a pretty sad reflection, but this is where this whole diversity issue gets conflated. We probably have the most representative government over this parliament, yet the most damage done to people of colour has been through this parliament as well. What a sad legacy that the first person of colour to be prime minister is going to leave, arguably punching down on his own.”

On his phone is a picture of his 66-year-old father, Muhammad, celebrating his birthday at their new home in Dubai, surrounded by grandchildren. Muhammad’s oxygen tank is behind him, out of shot – Rafiq’s father has chronic obstructive pulmonary disease, and the strain of upheaval has only worsened his condition. “We’re enjoying the time he’s got left,” says Rafiq. “But I’m going to have to live with the fact that me speaking out has accelerated the end of my dad’s life.” The irony is not lost on him that just over 20 years ago his father also relocated their family to keep them safe.

Rafiq was born in Karachi in 1991. His memories of Pakistan are mostly cricket-related. “I just remember coming home every day in a complete and utter mess from throwing myself around, blood and dust all over me.” He first witnessed the violence in the city when he was about nine years old, and went to investigate the unusual sounds coming from the street behind his building.

“The sun had gone down a bit so you had hundreds of kids playing volleyball and other games,” he says. “And there was a car driving past just shooting at them.” Some died in front of him; even today he doesn’t understand why. “It was something around caste and stuff, but I don’t really know.”

Then one of his father’s business partners was kidnapped. Muhammad had built up a successful property business, and its owners were now targets. “A phone call came: ‘We found the car; we think you should come down here.’” Rafiq’s father arrived at the scene to find the burned remains of his friend’s body.

In July 2001 the family sought asylum in the UK, and for two months, nine of them lived in two rooms in a London hotel. By October, they had been placed in a house in Barnsley. It’s worth noting, given all that has happened since, that Rafiq’s descriptions of that town are largely positive – a pleasant suburb, kind neighbours, playing with other local kids in the back garden. The young Rafiq already spoke English, and his passion for cricket helped him to integrate further. But he also discovered how privileged their previous life in Pakistan had been. When there were no new clothes for Eid, he writes in his book, he realised that “we could no longer just get whatever we wanted”.

His parents found their new life harder. Muhammad’s asylum seeker status meant he was unable to work. In the aftermath of 9/11 his long beard attracted attention and he was called “Bin Laden” in the street. One night the family called the police after bricks were thrown through their windows, and a decision was made to move back to Pakistan. The very next morning Rafiq received a letter telling him he had been picked to play in the Yorkshire Under-12s. “They only told me this when I spoke out,” says Rafiq. “That’s the first I knew that my cricket career was really the reason we end up staying in the UK.”

His was a truly promising talent. After making his first-team debut for Yorkshire as a teenager, Rafiq captained an England Under-19s side that included future England captains Joe Root, Ben Stokes and Jos Buttler (a thought that “gives me goosebumps even now”). In 2012 he became Yorkshire’s first Asian captain, as well as the youngest in their history, helping lead them to the final of the Twenty20 Cup. “I was in my element; it was me at my happiest,” he says. “Over the last few years I’ve not really talked about my cricket because I think when you start doing that it allows people to get away from the [racism] conversation … but I absolutely thrived on it.”

Two years later, aged only 23, he was dropped by the county and his playing career seemed over. Rafiq has repeated the story of that time – and his brief return from 2016 to 2018 – many, many times at tribunals, panels and independent commissions. What emerges from the retellings is the toxic nature of the Yorkshire dressing room, a place where “friendly banter” involved the exchange of racial slurs and nicknames, and where Rafiq could lose his first son, Alyaan, to a stillbirth and never receive a word of compassion from his employers or his teammates.

The drinking culture was another problem. In the book, Rafiq attributes some of his mental health struggles to the fact that he started drinking alcohol to fit in. “It’s the one thing I do regret,” he says now. “It wasn’t something I enjoyed … but I had this mad dream. I just wanted to play for England and I was prepared to do anything and everything for it.”

Cricket caused him to “end up living two different lives … South Asians, but especially Muslims, we have these two different cultures, one at home and one at work.” He says he’s often asked why he took so long to speak out about the bullying he suffered, and thinks back to the time when, as a 15-year-old, he was pinned down by a teammate in the back of a car and had red wine poured down his throat. “Imagine if I went home and told my dad. I’d never play cricket again.”

In the end it was family that saved him. He married Faryal in 2015, a match that had been arranged for them in Pakistan. “She’s been a rock,” he says. “To move countries, away from her family, to give up everything … I just hope the next part of my life I can give her some peace.” His parents and siblings, who had assumed he was living his dream life as a cricketer, protested in solidarity with him outside the gates of the Headingley ground. “When I spoke out, my dad said to me in Urdu: ‘I’ve seen you cry with blood tears. Don’t back down.’”

It’s clear he still has next to no trust in the institutions that run English cricket, or the steps they’re taking to tackle racism and improve inclusion. He rails against the measures he says have been taken to silence other victims – “NDAs, payouts, jobs for people that were going to speak out” – and is sceptical of “community and faith organisations” that he believes are telling cricket’s governing bodies what they want to hear, too reliant on their funding to hold them to account. He maintains that the reporting, safeguarding and regulatory practices in English cricket are not yet fit for purpose, and that increased diversity at board level makes no difference to the culture of the game.

You can understand the grounds for some of these doubts. This year Colin Graves, who chaired Yorkshire from 2007 to 2015, with 2012 to 2015 as executive chair, and had dismissed his claims of racism, returned to the position when the club was threatened with bankruptcy. There are those who argue that Graves was only able to do so because of the financial mess that was caused by the sacking of Yorkshire’s 16-strong coaching staff en masse in the aftermath of the scandal. The decision was taken by Lord Patel, the chair who was appointed to oversee a change in the club’s culture, but the dismissals were ultimately ruled unfair and cost £1.9m in legal payouts.

This is not a line to take with Rafiq, who even this month had a bitter row on X with a cricket journalist making a fairly measured observation on the issue. He says he has got better at disengaging from social media – he used to try to respond to everyone who trolled him – but clearly these subjects remain personal and raw. “I can be very reasonable,” he says. “But I can be very unreasonable as well.”

The refusal to compromise that has been, until now, his greatest strength, may be reaching its limits. Some of his closest allies, people he respects, have spoken of the need to move him into a less adversarial position. He says he won’t be a cheerleader for a system he doesn’t believe in. “I’m still the problem from a cricket point of view and it works better for cricket to turn me into a problem … and you know what? It’s fine by me. I never started this off to be popular.”

Few relationships from his playing days have endured: he feels the sport has rejected him. “Even those that supported me initially backed off when it got bigger. And it hurts me. Cricket’s the thing I love, and I don’t feel love from cricket.” He says he is particularly disappointed in Root, who testified that he never witnessed racism at Yorkshire: “I think he’s let himself down.” He has more sympathy for Gary Ballance, one of his teammates charged with using racist language, who has since retired from cricket after struggling with his mental health.

Rafiq met him for an in-person apology in 2022. “I saw a broken person. And this could have been stopped if more people didn’t just back off and think about themselves. Their silence not only has damaged me but it’s damaged the likes of Gary.”

The narrative of his time at Yorkshire has come to define him; you wonder whether the book’s publication can be the moment he moves on from it. There are chapters on reconciliation and allyship, as well as suggestions of practical solutions to the mess that cricket has found itself in. This is partly because, he says, he wanted to strike a positive note: “I didn’t want people to read it and fight me again.”

He says his future goal is to “support and challenge” the game. One project he’s hoping to develop is “an X Factor for cricket”, scouting for male and female talent in open auditions up and down the country, with a professional contract as the prize. “I want to use my platform to get the resources, the funds, to be able to provide opportunities to those that can’t afford to have opportunities.” He points to the work that Stormzy is doing, recently opening a new community football centre in Croydon.

He recognises that “there’s a lot of people looking at me to see what happens next”. For the past three years he has committed to every major battle and minor skirmish on this ground, but the critics, the doubters and the racist trolls take their toll. Does he have an endgame in mind for this fight?

“I don’t think there’s an endgame. Racism doesn’t go – it evolves, it changes,” he says. “But I’ve got a sportsperson’s mentality. I won’t give up.”

It’s Not Banter, It’s Racism by Azeem Rafiq will be published on 13 June (Trapeze, £22). To support the Guardian, buy your copy from guardianbookshop.com. Delivery charges may apply