The factors that account for the differences in the economic productivity of urban areas have remained difficult to measure and identify unambiguously. Here we show that a microscopic derivation of urban scaling relations for economic quantities vs. population, obtained from the consideration of social and infrastructural properties common to all cities, implies an effective model of economic output in the form of a Cobb-Douglas type production function. As a result we derive a new expression for the Total Factor Productivity (TFP) of urban areas, which is the standard measure of economic productivity per unit of aggregate production factors (labor and capital). Using these results we empirically demonstrate that there is a systematic dependence of urban productivity on city population size, resulting from the mismatch between the size dependence of wages and labor, so that in contemporary US cities productivity increases by about 11% with each doubling of their population. Moreover, deviations from the average scale dependence of economic output, capturing the effect of local factors, including history and other local contingencies, also manifest surprising regularities. Although, productivity is maximized by the combination of high wages and low labor input, high productivity cities show invariably high wages and high levels of employment relative to their size expectation. Conversely, low productivity cities show both low wages and employment. These results shed new light on the microscopic processes that underlie urban economic productivity, explain the emergence of effective aggregate urban economic output models in terms of labor and capital inputs and may inform the development of economic theory related to growth.