Johnson, 19, chose to join high-flyers Grays Athletic instead of the Shots this week after being made available for loan until the end of the season by his club Northampton Town.
Brown insists he was on the verge of bringing his young protégé back to the Rec in a bid to bolster his defence — only to be beaten by a last ditch move Grays boss Mark Stimson.
The Shots boss said: “I'm very disappointed, of course I am. It looked as though we had done a deal with Northampton but Brett decided he felt it would be a bit awkward coming back with people thinking he hadn't done it at Northampton.
“He's not a bad lad and I understand his reasons even though from an Aldershot manager's point of view, I'm extremely disappointed to have missed out.
“I felt the move happened a year too early for Brett and that's proved to be the case, but he's a talented young boy and I'm sure he can still make it.”
Brown's transfer disappointment continued this week when a tragic car crash and an injury to another player, all but ended his chances in luring No.1 target Jonny Dixon from Wycombe Wanderers.
The Shots boss seemed close to getting his man last week, but the shocking and tragic death of youngster Mark Philo in an accident at Wokingham on Saturday morning, coupled with a fresh injury sustained by striker Charlie Griffin, means that boss John Gorman is likely to want to keep hold of Dixon for the time being.
Brown added: “I speak to John Gorman on a regular basis but I don't know about Dixon just yet. John has had to suffer the death of the young lad Philo and with Griffin's injury, like so many other situations at the moment, any deal is dependent on the selling club.
“It is common knowledge though that I am after Dixon as I think he will be a fantastic asset to our club.”
With thanks to the Aldershot News.