Here is the Oxford English Dictionary on the term:
1853 ‘P. Paxton’ 205 By a unanimous vote, Judge G—— was elected to the bench and the ‘Mestang’ or ‘Kangaroo Court’ regularly organized.
1895 Apr. 718/2 The most interesting of these impromptu clubs is the one called in the vernacular the ‘Kangaroo Court’. It is found almost entirely in county jails.
1931 ‘D. Stiff’ 209 Kangaroo court, mock court held in jail for the purpose of forcing new prisoners to divide their money.
1935 A. J. Pollock 66/1 Kangaroo Court, a jail tribunal comprised of inmates which collects money from prisoners awaiting trial to supply the needy with tobacco, food and a few luxuries—its decision regarding disputes is final.
1966 14 Mar. 10/1 Shop stewards at Theale are to meet tomorrow to consider paying back the sums levied by a kangaroo court.
1971 20 Jan. 15/3 Citizens who live in the riotous areas [of N. Ireland] deserve protection from..kangaroo courts.
1973 C. Mullardiii. vii. 81 Such practices are surely more like those of a kangaroo court than those that the Race Relations Board should encourage.
Note the term mustang court in the first citation.That’s about animals too. There were not many kangaroos in the USA but according to Wikipedia there were a lot of Australians in California in the gold rush.