"Bruce Elliott" wrote in message <ocldp3$akt$1@newscl01ah.mathworks.com>...
> "dpb" wrote in message <ocjbvg$rsr$1@dont-email.me>...
> > "There is no free lunch". I have observed that categorical are slow
> > even with just some small toy datasets. I think it's likely that the
> > initial implementation is not optimum; whether one can expect
> > improvement in the near or relatively near term I don't know; don't know
> > where efficiency ranks any longer with TMW vis a vis new features;
> > surely seems the latter are the main emphasis.
> >
> > I'd document a a comparison test case and submit it as a bug
> > report/slash/enhancement request that illustrates the issue. Who knows,
> > you might just get a suggestion on workarounds or at least it puts TMW
> > on notice of "Houston, we have a problem!".
> >
> > BTW, TABLE in general is slow vis a vis direct lookup in arrays albeit
> > at a large convenience.
>
> Thanks for your response, dpb. I was kind of fearing that what you said might be the case. I've started using tables and categorical data only fairly recently, and so far I've been pleased with the functionality. I had not anticipated the performance hit, however, so it's good to know about. In some cases memory might be cheaper than time, so the costs could outweigh the benefits.
Categorical data and tables in general use more memory and are less performant than the corresponding implementation using simple arrays. So the actual tradeoff here is not memory vs. performance, but rather maintainability and development time/cost vs. performance.
Yair Altman
http://UndocumentedMatlab.com
Author: "Accelerating MATLAB Performance" (CRC Press, 2014)
> "dpb" wrote in message <ocjbvg$rsr$1@dont-email.me>...
> > "There is no free lunch". I have observed that categorical are slow
> > even with just some small toy datasets. I think it's likely that the
> > initial implementation is not optimum; whether one can expect
> > improvement in the near or relatively near term I don't know; don't know
> > where efficiency ranks any longer with TMW vis a vis new features;
> > surely seems the latter are the main emphasis.
> >
> > I'd document a a comparison test case and submit it as a bug
> > report/slash/enhancement request that illustrates the issue. Who knows,
> > you might just get a suggestion on workarounds or at least it puts TMW
> > on notice of "Houston, we have a problem!".
> >
> > BTW, TABLE in general is slow vis a vis direct lookup in arrays albeit
> > at a large convenience.
>
> Thanks for your response, dpb. I was kind of fearing that what you said might be the case. I've started using tables and categorical data only fairly recently, and so far I've been pleased with the functionality. I had not anticipated the performance hit, however, so it's good to know about. In some cases memory might be cheaper than time, so the costs could outweigh the benefits.
Categorical data and tables in general use more memory and are less performant than the corresponding implementation using simple arrays. So the actual tradeoff here is not memory vs. performance, but rather maintainability and development time/cost vs. performance.
Yair Altman
http://UndocumentedMatlab.com
Author: "Accelerating MATLAB Performance" (CRC Press, 2014)
