Tell me if I am right, just rephrasing your question in this way :Given four points that belong to the surface of an sphere determine the volume of the tetrahedrom forms with these vertexes.

It is that your case, as you probably remember the volume of any tetrahedrom is one sixth of the volume of parellelopiped formed with three concurrent edges, which as you also remenber from vector calculus it is the product A*(BxC) . Where symbol * stands for scalar product and x for vector product. And A, B and C means the vector PA, PB and PC naming P the point on the spherical surface taking as origin.

Having said that what is now to be done it is to translate the spherical coordinates of the four points into rectangular coordinates.

Xa = Rcos(lat)cos(long) ;

Ya = Rcos(lat)sin(long) ;

Za = Rsin(lat) ;

and same for the other points,

Now substracting the coordinates of P you will obtain the components of vector A, B, and C

A is : (Xa-Xp)i +(Ya-Yp)j +(Za-Zp)k

B is : (Xb-Xp)i +(Yb-Yp)j +(Zb-Zp)k

C is : (Xc-Xp)i +(Yc-Yp)j +(Zc-Zp)k

Where

R stands for the sphere radius, lat means latitude and long means longitude or you can use altitude and azimuth

so the volume is 1/6 of the determinant

Ax Ay Az

Bx By Bz

Cx Cy Cz

Since you want the algorithm related to the spherical coordinates leave indicated in the formula the latitude and longitude of the points.