The average of the rank of elliptic curves over rational numbers in a ``natural'' family is expected to be 1/2. For example, Goldfeld conjectured that the average of analytic ranks of the quadratic twist family of an elliptic curve over rational numbers is 1/2. In this talk, we will introduce machinery which gives an upper bound of the average of analytic ranks of a family of elliptic curves. To run the machinery, we need to know the probability that an elliptic curve in the family has good/multiplicative/additive reduction (actually we need something more) and use trace formulas. Using the machinery on the set of all elliptic curves over rationals and the set of elliptic curves with a given torsion subgroup respectively, we can compute an upper bound of the n-th moment of the average. This is the first result on an upper bound of the average of the family of elliptic curves with a fixed torsion group, as far as we know. This is joint work with Peter J. Cho.