table.bulk_fetch() with meta attributes?

Daniel Manocchio

Hi Guys,

is there any way to have meta-attributes populated using the table.bulk_fetch() function without passing all their attribute definitions?

Calling bulk_fetch() with a list of (string) names only populates the attribute values, but not the meta-attribute values. Hence, I create a list of table attribute definitions containing both attributes and meta-attributes. When this list is used for calling bulk_fetch(), also the named meta-attributes are populated.

Now, I'm working on a project in which I generally need all the meta-attributes of a set of attributes. Building this list by table attribute definitiones is quite inconvenient. Hence I'm wondering whether there is a more elegant solution for this use case.

Ideas, anyone? :-)

Thanks in advance,
Daniel

Andy_Grigg

Understood. I'll add the enhancement to the development backlog. We'll try to get to it (or something that satisfies the same use case) before the next release.

Andy_Grigg

If you call Table.bulk_fetch() without any attributes specified, the scripting toolkit will fetch data for all attributes, meta-attributes, and pseudoattributes. If you need all attributes and meta-attributes, then this might be a good option.

If not, we can look at an enhancement to be able to specify that all meta-attributes are included for the specified attributes in a bulk_fetch operation. This would probably be via an additional optional keyword argument on the bulk_fetch method.

Please let me know what you think about the two options above.

Daniel Manocchio

Hi Andy,
thanks for your quick reply. Yeah, I've already used bulk_fetch() that way. However, for tables with a huge number of attributes and/or a higher number of records being processed, this becomes quite imperformant - at least on our system. So an additional argument like a boolean "include_metas=True" would be really great in some use cases

Daniel Manocchio

Cool, thanks a lot!

Andy_Grigg

Just to follow up on this, the solution we've chosen for the next release is to document the one or two lines of code necessary to add all meta-attributes to the bulk_fetch request. I don't want to go too much into the reasons here, but essentially it would add a lot of complexity to both the Scripting Toolkit itself and the bulk_fetch interface to do this properly.

The documentation should be much clearer now though, and provides the code snippet required in a copy-and-paste-able form.

Daniel Manocchio

Hi Andy,
thanks for the update! I guess these "one or two lines of code" just extend the list of attribute definitions with the metas of each attribute? Anyway, I'm excited

Andy_Grigg

That's exactly it, here's the snippet:

table = db.get_table("MaterialUniverse")
attribute_names = ["Density", "Price", "Young's modulus"]
print("Fetch three attributes by name")
table.bulk_fetch(records, attributes=attribute_names)

attribute_definitions = [table.attributes[name] for name in attribute_names]
print("Fetch three attributes by AttributeDefinition")
table.bulk_fetch(records, attributes=attribute_definitions)

attr_and_meta_definitions = attribute_definitions + [meta_attr
                                                     for attr in attribute_definitions
                                                     for meta_attr in attr.meta_attributes.values()]
print("Fetch three attributes and their meta-attributes")
table.bulk_fetch(records, attributes=attr_and_meta_definitions)

attr_meta_and_pseudo_definitions = attr_and_meta_definitions + mpy.RecordProperties.all()
print("Fetch three attributes, their meta-attributes, and all pseudo-attributes")
table.bulk_fetch(records, attributes=attr_meta_and_pseudo_definitions)

Daniel Manocchio

Hi Andy,
ok then, seems like something that'll find its way as function in my personal toolbox

Thanks & bests,
Daniel